# pypi_packages

name: datasette-pins
summary: Pin databases, tables, and other items to the Datasette homepage
classifiers: ["Framework :: Datasette"]
author: Alex Garcia
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-pins/
project_url: https://pypi.org/project/datasette-pins/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-pins/0.1a7/
requires_python: >=3.10
version: 0.1a7
description:
  # datasette-pins

  [![PyPI](https://img.shields.io/pypi/v/datasette-pins.svg)](https://pypi.org/project/datasette-pins/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-pins?include_prereleases&label=changelog)](https://github.com/datasette/datasette-pins/releases)
  [![Tests](https://github.com/datasette/datasette-pins/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-pins/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-pins/blob/main/LICENSE)

  Pin databases, tables, queries, and more to the Datasette homepage!

  <img src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2024/datasette-pins/hero.png"/>

  ## Installation

  `datasette-pins` requires a recent 1.0 alpha version of Datasette to work.

  ```bash
  pip install datasette>=1.0a13
  ```
  Afterwards, install this plugin in the same environment as Datasette.

  ```bash
  datasette install datasette-pins
  ```

  ## Configuration

  `datasette-pins` stores pins in the [internal database](https://docs.datasette.io/en/latest/internals.html#datasette-s-internal-database). This database runs in-memory by default, which means your pins will be lost if you restart the server. To avoid that, start Datasette with the `--internal` option to specify where that database should be stored on disk:

  ```bash
  datasette mydatabase.db --internal internal.db
  ```
  `datasette-pins` has two permissions `datasette-pins-write` and
  `datasette-pins-read`. Actors with the `datasette-pins-write` permissions can
  pin and reorder items, while actors with `datasette-pins-read` permissions can
  only view pinned items.

  Here's an example
  [`datasette.yaml` file](https://docs.datasette.io/en/latest/configuration.html#datasette-yaml-reference)
  where all actors can view pins, but only the `root` actor can pin items:

  ```yaml
  permissions:
    datasette-pins-read:
      id: "*"
      unauthenticated: true
    datasette-pins-write:
      id: "root"
  ```

  ## Usage

  Once logged in, the `root` use will see new pin/unpin option under the database,
  table, and query actions menu:

  <img src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2024/datasette-pins/table-example.png"/>

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new
  virtual environment:

  ```bash
  cd datasette-pins
  python3 -m venv venv
  source venv/bin/activate
  ```

  Now install the dependencies and test dependencies:

  ```bash
  pip install -e '.[test]'
  ```

  To run the tests:

  ```bash
  pytest
  ```

  ### Building JavaScript from TypeScript

  This plugin includes TypeScript/TSX source files in `datasette_pins/frontend/` that need to be compiled to JavaScript. If you modify any `.ts` or `.tsx` files, you'll need to rebuild the JavaScript.

  First install the Node.js dependencies:

  ```bash
  npm install
  ```

  Then build the JavaScript using [just](https://github.com/casey/just):

  ```bash
  just js
  ```

  This uses esbuild to compile and minify the TypeScript files into `datasette_pins/static/`.

project_urls:
  {"CI": "https://github.com/datasette/datasette-pins/actions", "Changelog": "https://github.com/datasette/datasette-pins/releases", "Homepage": "https://github.com/datasette/datasette-pins", "Issues": "https://github.com/datasette/datasette-pins/issues"}

requires_dist:
  ["datasette>=1.0a21", "sqlite-migrate>=0.1b0", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "ruff; extra == \"test\""]


name: datasette-placekey
summary: SQL functions for working with placekeys
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-placekey
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-placekey/
project_url: https://pypi.org/project/datasette-placekey/
release_url: https://pypi.org/project/datasette-placekey/0.1/
requires_dist: ["datasette", "placekey", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1
description:
  # datasette-placekey

  [![PyPI](https://img.shields.io/pypi/v/datasette-placekey.svg)](https://pypi.org/project/datasette-placekey/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-placekey?include_prereleases&label=changelog)](https://github.com/simonw/datasette-placekey/releases)
  [![Tests](https://github.com/simonw/datasette-placekey/workflows/Test/badge.svg)](https://github.com/simonw/datasette-placekey/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-placekey/blob/main/LICENSE)

  SQL functions for working with [placekeys](https://www.placekey.io/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-placekey

  ## Usage

  The following SQL functions are exposed - [documentation here](https://placekey.github.io/placekey-py/placekey.html#module-placekey.placekey).

  ```sql
  select
    geo_to_placekey(33.0896104,129.7900839),
    placekey_to_geo('@6nh-nhh-kvf'),
    placekey_to_geo_latitude('@6nh-nhh-kvf'),
    placekey_to_geo_longitude('@6nh-nhh-kvf'),
    placekey_to_h3('@6nh-nhh-kvf'),
    h3_to_placekey('8a30d94e4c87fff'),
    placekey_to_geojson('@6nh-nhh-kvf'),
    placekey_to_wkt('@6nh-nhh-kvf'),
    placekey_format_is_valid('@6nh-nhh-kvf');
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-placekey
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-placekey/actions", "Changelog": "https://github.com/simonw/datasette-placekey/releases", "Homepage": "https://github.com/simonw/datasette-placekey", "Issues": "https://github.com/simonw/datasette-placekey/issues"}


name: datasette-plot
summary: Observable Plot Datasette plugin
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Alex Garcia
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-plot/
project_url: https://pypi.org/project/datasette-plot/
release_url: https://pypi.org/project/datasette-plot/0.1.0/
requires_python: >=3.8
version: 0.1.0
description:
  # datasette-plot

  [![PyPI](https://img.shields.io/pypi/v/datasette-plot.svg)](https://pypi.org/project/datasette-plot/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-plot?include_prereleases&label=changelog)](https://github.com/datasette/datasette-plot/releases)
  [![Tests](https://github.com/datasette/datasette-plot/workflows/Test/badge.svg)](https://github.com/datasette/datasette-plot/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-plot/blob/main/LICENSE)

  A Datasette plugin for making data visualizations with [Observable Plot](https://observablehq.com/plot/) .

  <img width="600px" src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/datasette-plot/hero.jpeg"/>

  ## Installation

  Install this plugin in the same environment as Datasette.

  ```bash
  datasette install datasette-plot
  ```

  ## Usage

  Once installed, table pages and SQL query pages will have a new "Show plot" button that will open the `datasette-plot`.

  <img width="600px" src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/datasette-plot/table-2.jpeg"/>

  Current, `datasette-plot` only supports a few [visualization marks](https://observablehq.com/plot/features/marks) from Plot, including dot, line, area, bar, and more.

  <video width=600 controls autoplay type="video/mp4" style="display: block; margin: 0 auto" >
    <source src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/datasette-plot/dsc-plot-02.mp4" type="video/mp4">
      Your browser does not support the video tag.
  </video>

  Use the `Link to this plot` URL to share visualization with others. The link will have your plot options encoded in the URL.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

  ```bash
  cd datasette-plot
  python3 -m venv venv
  source venv/bin/activate
  ```

  Now install the dependencies and test dependencies:

  ```bash
  pip install -e '.[test]'
  ```

  To run the tests:

  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-plot/actions", "Changelog": "https://github.com/datasette/datasette-plot/releases", "Homepage": "https://github.com/datasette/datasette-plot", "Issues": "https://github.com/datasette/datasette-plot/issues"}

requires_dist:
  ["datasette >=1.0a6", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "ruff ; extra == 'test'"]


name: datasette-plugin-template-demo
summary: Demonstrating https://github.com/simonw/datasette-plugin
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-plugin-template-demo
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-plugin-template-demo/
project_url: https://pypi.org/project/datasette-plugin-template-demo/
release_url: https://pypi.org/project/datasette-plugin-template-demo/0.1/
version: 0.1
description:
  # datasette-plugin-template-demo

  [![PyPI](https://img.shields.io/pypi/v/datasette-plugin-template-demo.svg)](https://pypi.org/project/datasette-plugin-template-demo/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-plugin-template-demo?label=changelog)](https://github.com/simonw/datasette-plugin-template-demo/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-plugin-template-demo/blob/master/LICENSE)

  Demonstrating https://github.com/simonw/datasette-plugin

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-plugin-template-demo

  ## Usage

  Usage instructions go here.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-plugin-template-demo
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-plugin-template-demo/actions", "Changelog": "https://github.com/simonw/datasette-plugin-template-demo/releases", "Homepage": "https://github.com/simonw/datasette-plugin-template-demo", "Issues": "https://gitlab.com/simonw/datasette-plugin-template-demo/issues"}

requires_dist:
[cell repeats an earlier one in this file: 104 characters]


name: datasette-ports
summary: Find all currently running Datasette instances and list their ports
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-ports/
project_url: https://pypi.org/project/datasette-ports/
release_url: https://pypi.org/project/datasette-ports/0.3/
requires_dist: ["click", "httpx"]
requires_python: >=3.10
version: 0.3
description:
  # datasette-ports

  [![PyPI](https://img.shields.io/pypi/v/datasette-ports.svg)](https://pypi.org/project/datasette-ports/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-ports?include_prereleases&label=changelog)](https://github.com/datasette/datasette-ports/releases)
  [![Tests](https://github.com/datasette/datasette-ports/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-ports/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-ports/blob/main/LICENSE)

  Find all currently running Datasette instances on your machine and list their ports, databases, versions and plugins.

  ## Installation

  Install this as a plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-ports
  ```

  Or run it directly using `uvx`:
  ```bash
  uvx datasette-ports
  ```
  Or install it as a standalone tool:
  ```bash
  uv tool install datasette-ports
  datasette-ports
  ```
  ## Usage

  If installed as a Datasette plugin:

  ```bash
  datasette ports
  ```

  Or as a standalone command:

  ```bash
  datasette-ports
  ```
  This uses `lsof` to find all Python processes listening on TCP ports, then probes each one to check if it is a Datasette instance. For each instance found it displays the URL, Datasette version, attached databases and installed plugins.

  Example output:
  ```
  http://127.0.0.1:8007/ - v1.0a26
    Directory: /Users/simon/dev/blog
    Databases:
      simonwillisonblog: /Users/simon/dev/blog/simonwillisonblog.db
    Plugins:
      datasette-llm
      datasette-secrets
  http://127.0.0.1:8001/ - v1.0a26
    Directory: /Users/simon/dev/creatures
    Databases:
      creatures: /tmp/creatures.db
    Plugins:
      datasette-extract
      datasette-llm
      datasette-secrets
  http://127.0.0.1:8900/ - v0.65.2
    Databases:
      logs: /Users/simon/Library/Application Support/io.datasette.llm/logs.db
  http://0.0.0.0:8014/ - v1.0a26
    Directory: /Users/simon/dev/datasette
    Databases:
      content: /Users/simon/dev/datasette/content.db
      trees: /Users/simon/dev/datasette/trees.db
      _internal
    Plugins:
      datasette-llm
      datasette-visible-internal-db
  ```

  Database paths come from each instance's `/-/databases.json` endpoint. Where the process working directory can be determined (via `/proc/<pid>/cwd` on Linux or `lsof` on macOS), relative paths are resolved to absolute ones. On platforms where the working directory cannot be determined the `Directory:` line is omitted and paths are shown as reported by Datasette.

  ### JSON output

  Use `--json` to get machine-readable output:

  ```bash
  datasette ports --json
  # or
  datasette-ports --json
  ```
  ```json
  [
    {
      "url": "http://127.0.0.1:8007/",
      "host": "127.0.0.1",
      "port": 8007,
      "pid": 42373,
      "cwd": "/Users/simon/dev/blog",
      "version": "1.0a26",
      "databases": [
        {"name": "simonwillisonblog", "path": "/Users/simon/dev/blog/simonwillisonblog.db"}
      ],
      "plugins": [
        "datasette-llm",
        "datasette-secrets"
      ]
    },
    {
      "url": "http://127.0.0.1:8900/",
      "host": "127.0.0.1",
      "port": 8900,
      "pid": 12345,
      "cwd": null,
      "version": "0.65.2",
      "databases": [
        {"name": "logs", "path": "/Users/simon/Library/Application Support/io.datasette.llm/logs.db"}
      ],
      "plugins": []
    }
  ]
  ```

  ## Development

  To set up this project locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-ports
  uv run datasette-ports --help
  # Or:
  uv run datasette ports --help
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-ports/actions", "Changelog": "https://github.com/datasette/datasette-ports/releases", "Homepage": "https://github.com/datasette/datasette-ports", "Issues": "https://github.com/datasette/datasette-ports/issues"}


name: datasette-pretty-traces
summary: Prettier formatting for ?_trace=1 traces
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-pretty-traces/
project_url: https://pypi.org/project/datasette-pretty-traces/
release_url: https://pypi.org/project/datasette-pretty-traces/0.7/
requires_dist: ["datasette"]
requires_python: >=3.10
version: 0.7
description:
  # datasette-pretty-traces

  [![PyPI](https://img.shields.io/pypi/v/datasette-pretty-traces.svg)](https://pypi.org/project/datasette-pretty-traces/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-pretty-traces?include_prereleases&label=changelog)](https://github.com/simonw/datasette-pretty-traces/releases)
  [![Tests](https://github.com/simonw/datasette-pretty-traces/workflows/Test/badge.svg)](https://github.com/simonw/datasette-pretty-traces/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-pretty-traces/blob/main/LICENSE)

  Prettier formatting for `?_trace=1` traces

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-pretty-traces
  ```
  ## Usage

  Once installed, run Datasette using `--setting trace_debug 1`:
  ```bash
  datasette fixtures.db --setting trace_debug 1
  ```
  Then navigate to any page and add `?_trace=` to the URL:

      http://localhost:8001/?_trace=1

  The plugin will scroll you down the page to the visualized trace information.

  ## Demo

  You can try out the demo here:

  - [/?_trace=1](https://latest-with-plugins.datasette.io/?_trace=1) tracing the homepage
  - [/github/commits?_trace=1](https://latest-with-plugins.datasette.io/github/commits?_trace=1) tracing a table page

  ## Screenshot

  ![Screenshot showing the visualization produced by the plugin](https://user-images.githubusercontent.com/9599/145883732-a53accdd-5feb-4629-94cd-f73407c7943d.png)

  ## Development

  To set up this plugin locally, checkout the code and run the tests with `uv run pytest`:
  ```bash
  cd datasette-pretty-traces
  uv run pytest
  ```
  To try your development version of plugin run it like this:
  ```bash
  uv run datasette fixtures.db --setting trace_debug 1
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-pretty-traces/actions", "Changelog": "https://github.com/simonw/datasette-pretty-traces/releases", "Homepage": "https://github.com/simonw/datasette-pretty-traces", "Issues": "https://github.com/simonw/datasette-pretty-traces/issues"}


name: datasette-profiles-actors-from-ids
summary: Use datasette-profiles to return actors_from_ids
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-profiles-actors-from-ids/
project_url: https://pypi.org/project/datasette-profiles-actors-from-ids/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-profiles-actors-from-ids/0.1a0/
requires_python: >=3.9
version: 0.1a0
description:
  # datasette-profiles-actors-from-ids

  [![PyPI](https://img.shields.io/pypi/v/datasette-profiles-actors-from-ids.svg)](https://pypi.org/project/datasette-profiles-actors-from-ids/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-profiles-actors-from-ids?include_prereleases&label=changelog)](https://github.com/datasette/datasette-profiles-actors-from-ids/releases)
  [![Tests](https://github.com/datasette/datasette-profiles-actors-from-ids/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-profiles-actors-from-ids/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-profiles-actors-from-ids/blob/main/LICENSE)

  Use [datasette-profiles](https://github.com/datasette/datasette-profiles) to handle [actors_from_ids](https://docs.datasette.io/en/latest/plugin_hooks.html#actors-from-ids-datasette-actor-ids)

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-profiles-actors-from-ids
  ```
  ## Usage

  Once installed, calls to the [datasette.actors_from_ids()](https://docs.datasette.io/en/latest/internals.html#datasette-actors-from-ids) method will incorporate details from user profiles stored by the [datasette-profiles](https://github.com/datasette/datasette-profiles) plugin.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-profiles-actors-from-ids
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  python -m pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-profiles-actors-from-ids/actions", "Changelog": "https://github.com/datasette/datasette-profiles-actors-from-ids/releases", "Homepage": "https://github.com/datasette/datasette-profiles-actors-from-ids", "Issues": "https://github.com/datasette/datasette-profiles-actors-from-ids/issues"}

requires_dist:
  ["datasette", "datasette-profiles>=0.1a3", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]


name: datasette-profiles
summary: Editable user profile pages for Datasette
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-profiles/
project_url: https://pypi.org/project/datasette-profiles/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-profiles/0.1a5/
requires_dist: ["datasette>=1.0a21", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]
requires_python: >=3.10
version: 0.1a5
description:
  # datasette-profiles

  [![PyPI](https://img.shields.io/pypi/v/datasette-profiles.svg)](https://pypi.org/project/datasette-profiles/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-profiles?include_prereleases&label=changelog)](https://github.com/datasette/datasette-profiles/releases)
  [![Tests](https://github.com/datasette/datasette-profiles/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-profiles/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-profiles/blob/main/LICENSE)

  Editable user profile pages for Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-profiles
  ```
  ## Usage

  Adds the ability for users to view profiles and edit their profile, from options in the main application menu.

  Stores data in the internal database, so Datasette needs to be run with the `--internal internal.db` option.

  ## Plugin hook

  This plugin adds a plugin hook: `bottom_profile(datasette, request, profile_actor)`. Other plugins can use this to return HTML to be included at the bottom of the public user profile page - or return an `async def` function that returns that HTML.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-profiles
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  python -m pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-profiles/actions", "Changelog": "https://github.com/datasette/datasette-profiles/releases", "Homepage": "https://github.com/datasette/datasette-profiles", "Issues": "https://github.com/datasette/datasette-profiles/issues"}


name: datasette-proxy-url
summary: Proxy a URL through a Datasette instance
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-proxy-url/
project_url: https://pypi.org/project/datasette-proxy-url/
release_url: https://pypi.org/project/datasette-proxy-url/0.1.1/
requires_python: >=3.8
version: 0.1.1
description:
  # datasette-proxy-url

  [![PyPI](https://img.shields.io/pypi/v/datasette-proxy-url.svg)](https://pypi.org/project/datasette-proxy-url/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-proxy-url?include_prereleases&label=changelog)](https://github.com/datasette/datasette-proxy-url/releases)
  [![Tests](https://github.com/datasette/datasette-proxy-url/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-proxy-url/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-proxy-url/blob/main/LICENSE)

  Proxy a URL through a Datasette instance

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-proxy-url
  ```
  ## Usage

  This plugin can be configured to set specific paths within Datasette to proxy content from another URL.

  Configure the plugin like this:
  ```json
  {
      "plugins": {
          "datasette-proxy-url": {
              "paths": [
                  {"path": "/proxy", "backend": "http://example.com/"},
              ]
          }
      }
  }
  ```
  Now any request to `/proxy` will serve the HTML content from `http://example.com/`.

  The `content-type` response header and the content body will be passed through. All other headers are currently ignored.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-proxy-url
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-proxy-url/actions", "Changelog": "https://github.com/datasette/datasette-proxy-url/releases", "Homepage": "https://github.com/datasette/datasette-proxy-url", "Issues": "https://github.com/datasette/datasette-proxy-url/issues"}

requires_dist:
  ["datasette", "httpx", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-httpx ; extra == 'test'", "datasette-test ; extra == 'test'"]


name: datasette-psutil
summary: Datasette plugin adding a /-/psutil debugging endpoint
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-psutil
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-psutil/
project_url: https://pypi.org/project/datasette-psutil/
project_urls: {"Homepage": "https://github.com/simonw/datasette-psutil"}
release_url: https://pypi.org/project/datasette-psutil/0.2/
version: 0.2
description:
  # datasette-psutil

  [![PyPI](https://img.shields.io/pypi/v/datasette-psutil.svg)](https://pypi.org/project/datasette-psutil/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-psutil.svg?style=svg)](https://circleci.com/gh/simonw/datasette-psutil)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-psutil/blob/master/LICENSE)

  Datasette plugin adding a `/-/psutil` debugging endpoint

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-psutil

  ## Usage

  Visit `/-/psutil` on your Datasette instance to see various information provided by [psutil](https://psutil.readthedocs.io/).

  ## Example

  You can visit a live demo here: https://datasette-psutil-demo-j7hipcg4aq-uw.a.run.app/-/psutil

  Sample output:
  ```
  process.open_files()
    popenfile(path='/tmp/fixtures.db', fd=6)

  process.connections()
    pconn(fd=5, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='169.254.8.130', port=8080), raddr=addr(ip='169.254.8.129', port=52621), status='ESTABLISHED')
    pconn(fd=3, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='0.0.0.0', port=8080), raddr=(), status='LISTEN')

  process.memory_info()
    pmem(rss=56827904, vms=242540544, shared=0, text=0, lib=0, data=0, dirty=0)

  process.cmdline()
    '/usr/local/bin/python'
    '/usr/local/bin/datasette'
    'serve'
    '--host'
    '0.0.0.0'
    '-i'
    'fixtures.db'
    '--cors'
    '--inspect-file'
    'inspect-data.json'
    '--port'
    '8080'

  process.parents()
    psutil.Process(pid=1, name='sh', started='23:19:29')

  process.threads()
    pthread(id=2, user_time=7.27, system_time=3.99)
    pthread(id=4, user_time=0.01, system_time=0.0)
    pthread(id=5, user_time=0.0, system_time=0.0)
    pthread(id=6, user_time=0.0, system_time=0.0)
    pthread(id=7, user_time=0.0, system_time=0.0)
    pthread(id=8, user_time=0.0, system_time=0.0)

  psutil.getloadavg()
    (0.0, 0.0, 0.0)

  psutil.cpu_times(True)
    scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)
    scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)

  psutil.virtual_memory()
    svmem(total=2147483648, available=2093080576, percent=2.5, used=31113216, free=2093080576, active=42860544, inactive=11513856, buffers=0, cached=23289856, shared=262144, slab=0)

  list(psutil.process_iter())
    psutil.Process(pid=1, name='sh', started='23:19:29')
    psutil.Process(pid=2, name='datasette', started='23:19:29')
  ```

requires_dist:
  ["datasette (>=0.44)", "psutil", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-public
summary: Make specific Datasette tables visible to the public
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-public
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-public/
project_url: https://pypi.org/project/datasette-public/
release_url: https://pypi.org/project/datasette-public/0.2.3/
requires_dist: ["datasette>=0.63", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]
requires_python: >=3.7
version: 0.2.3
description:
  # datasette-public

  [![PyPI](https://img.shields.io/pypi/v/datasette-public.svg)](https://pypi.org/project/datasette-public/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-public?include_prereleases&label=changelog)](https://github.com/simonw/datasette-public/releases)
  [![Tests](https://github.com/simonw/datasette-public/workflows/Test/badge.svg)](https://github.com/simonw/datasette-public/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-public/blob/main/LICENSE)

  Make specific Datasette tables visible to the public

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-public

  ## Usage

  Any tables listed in the `_public_tables` table will be visible to the public, even if the rest of the Datasette instance does not allow anonymous access.

  The root user (and any user with the new `public-tables` permission) will get a new option in the table action menu allowing them to toggle a table between public and private.

  Installing this plugin also causes `allow-sql` permission checks to fall back to checking if the user has access to the entire database. This is to avoid users with access to a single public table being able to access data from other tables using the `?_where=` query string parameter.

  ## Configuration

  This plugin creates a new table in one of your databases called `_public_tables`.

  This table defaults to being created in the first database passed to Datasette.

  To create it in a different named database, use this plugin configuration:

  ```json
  {
    "plugins": {
      "datasette-public": {
        "database": "database_to_create_table_in"
      }
    }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-public
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-public/actions", "Changelog": "https://github.com/simonw/datasette-public/releases", "Homepage": "https://github.com/simonw/datasette-public", "Issues": "https://github.com/simonw/datasette-public/issues"}


name: datasette-publish-azure
summary: Publish Datasette instances to Azure Functions
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-publish-azure
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-publish-azure/
project_url: https://pypi.org/project/datasette-publish-azure/
release_url: https://pypi.org/project/datasette-publish-azure/0.1a0/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1a0
description:
  # datasette-publish-azure

  [![PyPI](https://img.shields.io/pypi/v/datasette-publish-azure.svg)](https://pypi.org/project/datasette-publish-azure/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-publish-azure?include_prereleases&label=changelog)](https://github.com/simonw/datasette-publish-azure/releases)
  [![Tests](https://github.com/simonw/datasette-publish-azure/workflows/Test/badge.svg)](https://github.com/simonw/datasette-publish-azure/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-publish-azure/blob/main/LICENSE)

  Publish Datasette instances to Azure Functions

  **UNDER CONSTRUCTION**: This plugin does not work yet!

  See [simonw/azure-functions-datasette](https://github.com/simonw/azure-functions-datasette) for the prototype.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-publish-azure

  ## Usage

  Usage instructions will go here.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-publish-azure
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-publish-azure/actions", "Changelog": "https://github.com/simonw/datasette-publish-azure/releases", "Homepage": "https://github.com/simonw/datasette-publish-azure", "Issues": "https://github.com/simonw/datasette-publish-azure/issues"}


name: datasette-publish-fly
summary: Datasette plugin for publishing data using Fly
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-publish-fly
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-publish-fly/
project_url: https://pypi.org/project/datasette-publish-fly/
release_url: https://pypi.org/project/datasette-publish-fly/1.3.1/
version: 1.3.1
description:
  # datasette-publish-fly

  [![PyPI](https://img.shields.io/pypi/v/datasette-publish-fly.svg)](https://pypi.org/project/datasette-publish-fly/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-publish-fly?include_prereleases&label=changelog)](https://github.com/simonw/datasette-publish-fly/releases)
  [![Tests](https://github.com/simonw/datasette-publish-fly/workflows/Test/badge.svg)](https://github.com/simonw/datasette-publish-fly/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-publish-fly/blob/main/LICENSE)

  [Datasette](https://datasette.io/) plugin for deploying Datasette instances to [Fly.io](https://fly.io/).

  Project background: [Using SQLite and Datasette with Fly Volumes](https://simonwillison.net/2022/Feb/15/fly-volumes/)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-publish-fly

  ## Deploying read-only data

  First, install the `flyctl` command-line tool by [following their instructions](https://fly.io/docs/getting-started/installing-flyctl/).

  Run `flyctl auth signup` to create an account there, or `flyctl auth login` if you already have one.

  You can now use `datasette publish fly` to publish one or more SQLite database files:

      datasette publish fly my-database.db --app="my-data-app"

  The argument you pass to `--app` will be used for the URL of your application: `my-data-app.fly.dev`.

  To update an application, run the publish command passing the same application name to the `--app` option.

  Fly have [a free tier](https://fly.io/docs/about/pricing/#free-allowances), beyond which they will charge you monthly for each application you have live.  Details of their pricing can be [found on their site](https://fly.io/docs/pricing/).

  Your application will be deployed at `https://your-app-name.fly.io/` - be aware that it may take several minutes to start working the first time you deploy it.

  ## Using Fly volumes for writable databases

  Fly [Volumes](https://fly.io/docs/reference/volumes/) provide persistant disk storage for Fly applications. Volumes can be 1GB or more in size and the Fly free tier includes 3GB of volume space.

  Datasette plugins such as [datasette-uploads-csvs](https://datasette.io/plugins/datasette-upload-csvs) and [datasette-tiddlywiki](https://datasette.io/plugins/datasette-tiddlywiki) can be deployed to Fly and store their mutable data in a volume.

  > :warning: **You should only run a single instance of your application** if your database accepts writes. Fly has excellent support for running multiple instances in different geographical regions, but `datasette-publish-fly` with volumes is not yet compatible with that model. You should probably [use Fly PostgreSQL instead](https://fly.io/blog/globally-distributed-postgres/).

  Here's how to deploy `datasette-tiddlywiki` with authentication provided by `datasette-auth-passwords`.

  First, you'll need to create a root password hash to use to sign into the instance.

  You can do that by installing the plugin and running the `datasette hash-password` command, or by using [this hosted tool](https://datasette-auth-passwords-demo.datasette.io/-/password-tool).

  The hash should look like `pbkdf2_sha256$...` - you'll need this for the next step.

  In this example we're also deploying a read-only database called `content.db`.

  Pick a name for your new application, then run the following:

      datasette publish fly \
      content.db \
      --app your-application-name \
      --create-volume 1 \
      --create-db tiddlywiki \
      --install datasette-auth-passwords \
      --install datasette-tiddlywiki \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...'

  This will create the new application, deploy the `content.db` read-only database, create a 1GB volume for that application, create a new database in that volume called `tiddlywiki.db`, then install the two plugins and configure the password you specified.

  ### Updating applications that use a volume

  Once you have deployed an application using a volume, you can update that application without needing the `--create-volume` or `--create-db` options. To add the [datasette-graphq](https://datasette.io/plugins/datasette-graphql) plugin to your deployed application you would run the following:

      datasette publish fly \
      content.db \
      --app your-application-name \
      --install datasette-auth-passwords \
      --install datasette-tiddlywiki \
      --install datasette-graphql \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \

  Since the application name is the same you don't need the `--create-volume` or `--create-db` options - these are persisted automatically between deploys.

  You do need to specify the full list of plugins that you want to have installed, and any plugin secrets.

  You also need to include any read-only database files that are part of the instance - `content.db` in this example - otherwise the new deployment will not include them.

  ### Advanced volume usage

  `datasette publish fly` will add a volume called `datasette` to your Fly application. You can customize the name using the `--volume name custom_name` option.

  Fly can be used to scale applications to run multiple instances in multiple regions around the world. This works well with read-only Datasette but is not currently recommended using Datasette with volumes, since each Fly replica would need its own volume and data stored in one instance would not be visible in others.

  If you want to use multiple instances with volumes you will need to switch to using the `flyctl` command directly. The `--generate-dir` option, described below, can help with this.

  ## Generating without deploying

  Use the `--generate-dir` option to generate a directory that can be deployed to Fly rather than deploying directly:

      datasette publish fly my-database.db \
        --app="my-generated-app" \
        --generate-dir /tmp/deploy-this

  You can then manually deploy your generated application using the following:

      cd /tmp/deploy-this
      flyctl apps create my-generated-app
      flyctl deploy

  ## datasette publish fly --help

  <!-- [[[cog
  import cog
  from datasette import cli
  from click.testing import CliRunner
  runner = CliRunner()
  result = runner.invoke(cli.cli, ["publish", "fly", "--help"])
  help = result.output.replace("Usage: cli", "Usage: datasette")
  cog.out(
      "```\n{}```".format(help)
  )
  ]]] -->
  ```
  Usage: datasette publish fly [OPTIONS] [FILES]...

    Deploy an application to Fly that runs Datasette against the provided database
    files.

    Usage example:

        datasette publish fly my-database.db --app="my-data-app"

    Full documentation: https://datasette.io/plugins/datasette-publish-fly

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to
                                    publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g.
                                    main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at
                                    /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret <TEXT TEXT TEXT>...
                                    Secrets to pass to plugins, e.g. --plugin-
                                    secret datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as
                                    signed cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --spatialite                    Enable SpatialLite extension
    --region TEXT                   Fly region to deploy to, e.g sjc - see
                                    https://fly.io/docs/reference/regions/
    --create-volume INTEGER RANGE   Create and attach volume of this size in GB
                                    [x>=1]
    --create-db TEXT                Names of read-write database files to create
    --volume-name TEXT              Volume name to use
    -a, --app TEXT                  Name of Fly app to deploy  [required]
    -o, --org TEXT                  Name of Fly org to deploy to
    --generate-dir DIRECTORY        Output generated application files and stop
                                    without deploying
    --show-files                    Output the generated Dockerfile, metadata.json
                                    and fly.toml
    --setting SETTING...            Setting, see
                                    docs.datasette.io/en/stable/settings.html
    --crossdb                       Enable cross-database SQL queries
    --help                          Show this message and exit.
  ```
  <!-- [[[end]]] -->

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd datasette-publish-fly
      python -m venv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

  ### Integration tests

  The tests in `tests/test_integration.py` make actual calls to Fly to deploy a test application.

  These tests are skipped by default. If you have `flyctl` installed and configured, you can run the integration tests like this:

      pytest --integration -s

  The `-s` option here ensures that output from the deploys will be visible to you - otherwise it can look like the tests have hung.

  The tests will create applications on Fly that start with the prefix `publish-fly-temp-` and then delete them at the end of the run.

project_urls:
  {"CI": "https://github.com/simonw/datasette-publish-fly/actions", "Changelog": "https://github.com/simonw/datasette-publish-fly/releases", "Homepage": "https://github.com/simonw/datasette-publish-fly", "Issues": "https://github.com/simonw/datasette-publish-fly/issues"}

requires_dist:
  ["datasette >=0.60.2", "pytest ; extra == 'test'", "pytest-mock ; extra == 'test'", "cogapp ; extra == 'test'"]


name: datasette-publish-vercel
summary: Datasette plugin for publishing data using Vercel
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-publish-vercel
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-publish-vercel/
project_url: https://pypi.org/project/datasette-publish-vercel/
release_url: https://pypi.org/project/datasette-publish-vercel/0.14.2/
requires_dist: ["datasette (>=0.59)", "pytest ; extra == 'test'"]
version: 0.14.2
description:
  # datasette-publish-vercel

  [![PyPI](https://img.shields.io/pypi/v/datasette-publish-vercel.svg)](https://pypi.org/project/datasette-publish-vercel/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-publish-vercel?include_prereleases&label=changelog)](https://github.com/simonw/datasette-publish-vercel/releases)
  [![Tests](https://github.com/simonw/datasette-publish-vercel/workflows/Test/badge.svg)](https://github.com/simonw/datasette-publish-vercel/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-publish-vercel/blob/main/LICENSE)

  Datasette plugin for publishing data using [Vercel](https://vercel.com/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-publish-vercel

  ## Usage

  First, install the Vercel CLI tool by [following their instructions](https://vercel.com/download).

  Run `vercel login` to login to (or create) an account.

  Now you can use `datasette publish vercel` to publish your data:

      datasette publish vercel my-database.db --project=my-database

  The `--project` argument is required - it specifies the project name that should be used for your deployment. This will be used as part of the deployment's URL.

  ### Other options

  * `--no-prod` deploys to the project without updating the "production" URL alias to point to that new deployment. Without that option all deploys go directly to production.
  * `--debug` enables the Vercel CLI debug output.
  * `--token` allows you to pass a Now authentication token, rather than needing to first run `now login` to configure the tool. Tokens can be created in the Vercel web dashboard under Account Settings -> Tokens.
  * `--public` runs `vercel --public` to publish the application source code at `/_src` e.g. https://datasette-public.now.sh/_src and make recent logs visible at `/_logs` e.g. https://datasette-public.now.sh/_logs
  * `--generate-dir` - by default this tool generates a new Vercel app in a temporary directory, deploys it and then deletes the directory. Use `--generate-dir=my-app` to output the generated application files to a new directory of your choice instead. You can then deploy it by running `vercel` in that directory.
  * `--setting default_page_size 10` - use this to set Datasette settings, as described in [the documentation](https://docs.datasette.io/en/stable/settings.html). This is a replacement for the unsupported `--extra-options` option.

  ### Full help

  **Warning:** Some of these options are not yet implemented by this plugin. In particular, the following do not yet work:

  * `--extra-options` - use `--setting` described above instead.
  * `--plugin-secret`
  * `--version-note`

  ```
  $ datasette publish vercel --help

  Usage: datasette publish vercel [OPTIONS] [FILES]...

    Publish to https://vercel.com/

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g. main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret <TEXT TEXT TEXT>...
                                    Secrets to pass to plugins, e.g. --plugin-secret
                                    datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as signed
                                    cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --token TEXT                    Auth token to use for deploy
    --project PROJECT               Vercel project name to use  [required]
    --scope TEXT                    Optional Vercel scope (e.g. a team name)
    --no-prod                       Don't deploy directly to production
    --debug                         Enable Vercel CLI debug output
    --public                        Publish source with Vercel CLI --public
    --generate-dir DIRECTORY        Output generated application files and stop without
                                    deploying
    --generate-vercel-json          Output generated vercel.json file and stop without
                                    deploying
    --vercel-json FILENAME          Custom vercel.json file to use instead of generating
                                    one
    --setting SETTING...            Setting, see docs.datasette.io/en/stable/settings.html
    --crossdb                       Enable cross-database SQL queries
    --help                          Show this message and exit.
  ```
  ## Using a custom `vercel.json` file

  If you want to add additional redirects or similar to your Vercel configuration you may want to provide a custom `vercel.json` file.

  To do this, first generate a configuration file (without running a deploy) using the `--generate-vercel-json` option:

      datasette publish vercel my-database.db \
        --project=my-database \
        --generate-vercel-json > vercel.json

  You can now edit the `vercel.json` file that this creates to add your custom options.

  Then run the deploy using:

      datasette publish vercel my-database.db \
        --project=my-database \
        --vercel-json=vercel.json

  ## Setting a `DATASETTE_SECRET`

  Datasette uses [a secret string](https://docs.datasette.io/en/stable/settings.html#configuring-the-secret) for purposes such as signing authentication cookies. This secret is reset when the server restarts, which will sign out any users who are authenticated using a signed cookie.

  You can avoid this by generating a `DATASETTE_SECRET` secret string and setting that as a [Vercel environment variable](https://vercel.com/docs/concepts/projects/environment-variables). If you do this the secret will stay consistent and your users will not be signed out.

  ## Using this with GitHub Actions

  This plugin can be used together with [GitHub Actions](https://github.com/features/actions) to deploy Datasette instances automatically on new pushes to a repo, or on a schedule.

  The GitHub Actions runners already have the Vercel deployment tool installed. You'll need to create an API token for your account at [vercel.com/account/tokens](https://vercel.com/account/tokens), and store that as a secret in your GitHub repository called `VERCEL_TOKEN`.

  Make sure your workflow has installed `datasette` and `datasette-publish-vercel` using `pip`, then add the following step to your GitHub Actions workflow:
  ```
      - name: Deploy Datasette using Vercel
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
        run: |-
          datasette publish vercel mydb.db \
            --token $VERCEL_TOKEN \
            --project my-vercel-project
  ```
  You can see a full example of a workflow that uses Vercel in this way [in the simonw/til repository](https://github.com/simonw/til/blob/12b3f0d3679320cbeafa5df164bbc08ba703625d/.github/workflows/build.yml).

project_urls:
  {"CI": "https://github.com/simonw/datasette-publish-vercel/actions", "Changelog": "https://github.com/simonw/datasette-publish-vercel/releases", "Homepage": "https://github.com/simonw/datasette-publish-vercel", "Issues": "https://github.com/simonw/datasette-publish-vercel/issues"}


name: datasette-pyinstrument
summary: Use pyinstrument to analyze Datasette page performance
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-pyinstrument
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-pyinstrument/
project_url: https://pypi.org/project/datasette-pyinstrument/
release_url: https://pypi.org/project/datasette-pyinstrument/0.1/
requires_dist: ["datasette", "pyinstrument", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-pyinstrument

  [![PyPI](https://img.shields.io/pypi/v/datasette-pyinstrument.svg)](https://pypi.org/project/datasette-pyinstrument/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-pyinstrument?include_prereleases&label=changelog)](https://github.com/simonw/datasette-pyinstrument/releases)
  [![Tests](https://github.com/simonw/datasette-pyinstrument/workflows/Test/badge.svg)](https://github.com/simonw/datasette-pyinstrument/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-pyinstrument/blob/main/LICENSE)

  Use pyinstrument to analyze Datasette page performance

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-pyinstrument

  ## Usage

  Once installed, adding `?_pyinstrument=1` to any URL within Datasette will replace the output of that page with the pyinstrument profiler results for it.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-pyinstrument
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-pyinstrument/actions", "Changelog": "https://github.com/simonw/datasette-pyinstrument/releases", "Homepage": "https://github.com/simonw/datasette-pyinstrument", "Issues": "https://github.com/simonw/datasette-pyinstrument/issues"}


name: datasette-python
summary: Run a Python interpreter in the Datasette virtual environment
classifiers: ["License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-python/
project_url: https://pypi.org/project/datasette-python/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-python/0.1/
requires_dist: ["datasette", "pytest; extra == \"test\""]
version: 0.1
description:
  # datasette-python

  [![PyPI](https://img.shields.io/pypi/v/datasette-python.svg)](https://pypi.org/project/datasette-python/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-python?include_prereleases&label=changelog)](https://github.com/datasette/datasette-python/releases)
  [![Tests](https://github.com/datasette/datasette-python/workflows/Test/badge.svg)](https://github.com/datasette/datasette-python/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-python/blob/main/LICENSE)

  Run a Python interpreter in the Datasette virtual environment

  ## Installation

  Install this plugin in the same environment as [Datasette](https://datasette.io/).
  ```bash
  datasette install datasette-python
  ```
  ## Usage

  This plugin adds a new `python` command to Datasette. This executes Python in the same virtual environment as Datasette itself.

  You can use this to check the Python version

  ```bash
  datasette python --version
  # Should output 'Python 3.12.4' or similar
  ```
  Or  to run commands like `pip`:
  ```bash
  datasette python -m pip install httpx
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-python
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-python/actions", "Changelog": "https://github.com/datasette/datasette-python/releases", "Homepage": "https://github.com/datasette/datasette-python", "Issues": "https://github.com/datasette/datasette-python/issues"}


name: datasette-queries
summary: Save SQL queries in Datasette
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-queries/
project_url: https://pypi.org/project/datasette-queries/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-queries/0.1.2/
requires_python: >=3.9
version: 0.1.2
description:
  # datasette-queries

  [![PyPI](https://img.shields.io/pypi/v/datasette-queries.svg)](https://pypi.org/project/datasette-queries/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-queries?include_prereleases&label=changelog)](https://github.com/datasette/datasette-queries/releases)
  [![Tests](https://github.com/datasette/datasette-queries/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-queries/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-queries/blob/main/LICENSE)

  Save SQL queries in Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-queries
  ```
  ## Usage

  Users with the `datasette-queries` permission will see a new UI element on the results of a SQL query inviting them to save that query.

  Saved queries (treated as [canned queries](https://docs.datasette.io/en/stable/sql_queries.html#canned-queries)) are then shown as a list on the database page.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-queries
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  python -m pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-queries/actions", "Changelog": "https://github.com/datasette/datasette-queries/releases", "Homepage": "https://github.com/datasette/datasette-queries", "Issues": "https://github.com/datasette/datasette-queries/issues"}

requires_dist:
  ["datasette>=1.0a16", "sqlite-migrate", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]


name: datasette-query-assistant
summary: Query databases and tables with AI assistance
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-query-assistant/
project_url: https://pypi.org/project/datasette-query-assistant/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-query-assistant/0.1a5/
requires_python: >=3.10
version: 0.1a5
description:
  # datasette-query-assistant

  [![PyPI](https://img.shields.io/pypi/v/datasette-query-assistant.svg)](https://pypi.org/project/datasette-query-assistant/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-query-assistant?include_prereleases&label=changelog)](https://github.com/datasette/datasette-query-assistant/releases)
  [![Tests](https://github.com/datasette/datasette-query-assistant/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-query-assistant/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-query-assistant/blob/main/LICENSE)

  Query databases and tables with AI assistance

  **Early alpha**.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-query-assistant
  ```

  ## Configuration

  Set the model ID to be used by this plugin by adding this to your Datasette configuration:

  ```yaml
  plugins:
    datasette-query-assistant:
      model: openai/gpt-4.1-mini
      key_env_var: OPENAI_API_KEY
  ```
  To use the `openai/gpt-4.1-mini` model you'll need to install `llm-openai-plugin` in addition to `datasette-query-assistant`.

  If the model needs an API key, set the `key_env_var` to the name of an environment variable and ensure that environment variable is correctly set.

  You can use any model that is available via an [LLM plugin](https://llm.datasette.io/en/stable/plugins/directory.html)

  ## Usage

  Users with `execute-sql` permission will gain a database action menu item for "Query this database with AI assistance" which will let them ask a question and be redirected to a commented SQL query that will hopefully answer it.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-query-assistant
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```
  To re-generate the tests with refreshed examples from the API:
  ```bash
  pytest -x --record-mode=rewrite --inline-snapshot=fix
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-query-assistant/actions", "Changelog": "https://github.com/datasette/datasette-query-assistant/releases", "Homepage": "https://github.com/datasette/datasette-query-assistant", "Issues": "https://github.com/datasette/datasette-query-assistant/issues"}

requires_dist:
  ["datasette>=1.0a21", "llm>=0.22", "markdown2", "llm-openai-plugin; extra == \"test\"", "inline-snapshot; extra == \"test\"", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "pytest-recording; extra == \"test\"", "sqlite-utils; extra == \"test\""]


name: datasette-query-files
summary: Write Datasette canned queries as plain SQL files
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Chris Amico
description_content_type: text/markdown
home_page: https://github.com/eyeseast/datasette-query-files
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-query-files/
project_url: https://pypi.org/project/datasette-query-files/
release_url: https://pypi.org/project/datasette-query-files/0.1.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1.1
description:
  # datasette-query-files

  [![PyPI](https://img.shields.io/pypi/v/datasette-query-files.svg)](https://pypi.org/project/datasette-query-files/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/datasette-query-files?include_prereleases&label=changelog)](https://github.com/eyeseast/datasette-query-files/releases)
  [![Tests](https://github.com/eyeseast/datasette-query-files/workflows/Test/badge.svg)](https://github.com/eyeseast/datasette-query-files/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/datasette-query-files/blob/main/LICENSE)

  Write Datasette canned queries as plain SQL files.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-query-files

  Or using `pip` or `pipenv`:

      pip install datasette-query-files
      pipenv install datasette-query-files

  ## Usage

  This plugin will look for [canned queries](https://docs.datasette.io/en/stable/sql_queries.html#canned-queries) in the filesystem, in addition any defined in metadata.

  Let's say you're working in a directory called `project-directory`, with a database file called `my-project.db`. Start by creating a `queries` directory with a `my-project` directory inside it. Any SQL file inside that `my-project` folder will become a canned query that can be run on the `my-project` database. If you have a `query-name.sql` file and a `query-name.json` (or `query-name.yml`) file in the same directory, the JSON file will be used as query metadata.

  ```
  project-directory/
    my-project.db
    queries/
      my-project/
        query-name.sql # a query
        query-name.yml # query metadata
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-query-files
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/eyeseast/datasette-query-files/actions", "Changelog": "https://github.com/eyeseast/datasette-query-files/releases", "Homepage": "https://github.com/eyeseast/datasette-query-files", "Issues": "https://github.com/eyeseast/datasette-query-files/issues"}


name: datasette-query-history
summary: Datasette plugin that keeps a list of the queries you've run and lets you rerun them.
classifiers: []
author: Bret Walker
description_content_type: text/markdown
home_page: https://github.com/bretwalker/datasette-query-history
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-query-history/
project_url: https://pypi.org/project/datasette-query-history/
release_url: https://pypi.org/project/datasette-query-history/0.2.3/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.2.3
description:
  # datasette-query-history

  [![PyPI](https://img.shields.io/pypi/v/datasette-query-history.svg)](https://pypi.org/project/datasette-query-history/)
  [![Changelog](https://img.shields.io/github/v/release/bretwalker/datasette-query-history?include_prereleases&label=changelog)](https://github.com/bretwalker/datasette-query-history/releases)
  [![Tests](https://github.com/bretwalker/datasette-query-history/workflows/Test/badge.svg)](https://github.com/bretwalker/datasette-query-history/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/bretwalker/datasette-query-history/blob/main/LICENSE)

  Datasette plugin that keeps a list of the queries you've run and lets you rerun them.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-query-history

  ## Usage

  Click the `Query History` button on the SQL editor page to see previous queries.  
  Click the ⬆︎ button to replace the current query with a previous query.  
  Click the `Clear Query History` button to clear the list previous queries.

  <img src="https://raw.githubusercontent.com/bretwalker/datasette-query-history/main/docs/datasette-query-history-example1.png" width="350px" alt="Screenshot of plugin">

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-query-history
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/bretwalker/datasette-query-history/actions", "Changelog": "https://github.com/bretwalker/datasette-query-history/releases", "Homepage": "https://github.com/bretwalker/datasette-query-history", "Issues": "https://github.com/bretwalker/datasette-query-history/issues"}


name: datasette-query-links
summary: Turn SELECT queries returned by a query into links to execute them
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-query-links
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-query-links/
project_url: https://pypi.org/project/datasette-query-links/
release_url: https://pypi.org/project/datasette-query-links/0.1.2/
requires_python: >=3.6
version: 0.1.2
description:
  # datasette-query-links

  [![PyPI](https://img.shields.io/pypi/v/datasette-query-links.svg)](https://pypi.org/project/datasette-query-links/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-query-links?include_prereleases&label=changelog)](https://github.com/simonw/datasette-query-links/releases)
  [![Tests](https://github.com/simonw/datasette-query-links/workflows/Test/badge.svg)](https://github.com/simonw/datasette-query-links/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-query-links/blob/main/LICENSE)

  Turn SELECT queries returned by a query into links to execute them

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-query-links

  ## Usage

  This is an experimental plugin, requiring Datasette  0.59a1 or higher.

  Any SQL query that returns a value that itself looks like a valid SQL query will be converted into a link to execute that SQL query when it is displayed in the Datasette interface.

  These links will only show for valid SQL query - if a SQL query would return an error it will not be turned into a link.

  ## Demo

  * [Here's an example query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++%27select+*+from+%5Bfacetable%5D%27+as+query%0D%0Aunion%0D%0Aselect%0D%0A++%27select+sqlite_version()%27%0D%0Aunion%0D%0Aselect%0D%0A++%27select+this+is+invalid+SQL+so+will+not+be+linked%27) showing the plugin in action.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-query-links
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-query-links/actions", "Changelog": "https://github.com/simonw/datasette-query-links/releases", "Homepage": "https://github.com/simonw/datasette-query-links", "Issues": "https://github.com/simonw/datasette-query-links/issues"}

requires_dist:
  ["datasette (>=0.59a1)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "beautifulsoup4 ; extra == 'test'"]


name: datasette-reconcile
summary: Adds a reconciliation API to Datasette.
author_email: David Kane <david@dkane.net>
description_content_type: text/markdown
package_url: https://pypi.org/project/datasette-reconcile/
project_url: https://pypi.org/project/datasette-reconcile/
release_url: https://pypi.org/project/datasette-reconcile/0.6.3/
requires_python: >=3.7
version: 0.6.3
classifiers:
  ["Development Status :: 4 - Beta", "Framework :: Datasette", "Programming Language :: Python", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy"]

description:
  # datasette-reconcile

  [![PyPI - Version](https://img.shields.io/pypi/v/datasette-reconcile.svg)](https://pypi.org/project/datasette-reconcile)
  [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/datasette-reconcile.svg)](https://pypi.org/project/datasette-reconcile)
  [![Changelog](https://img.shields.io/github/v/release/drkane/datasette-reconcile?include_prereleases&label=changelog)](https://github.com/drkane/datasette-reconcile/releases)
  [![Tests](https://github.com/drkane/datasette-reconcile/workflows/Test/badge.svg)](https://github.com/drkane/datasette-reconcile/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/drkane/datasette-reconcile/blob/main/LICENSE)

  Adds a reconciliation API endpoint to [Datasette](https://docs.datasette.io/en/stable/), based on the [Reconciliation Service API](https://reconciliation-api.github.io/specs/latest/) specification.

  The reconciliation API is used to match a set of strings to their correct identifiers, to help with disambiguation and consistency in large datasets. For example, the strings "United Kingdom", "United Kingdom of Great Britain and Northern Ireland" and "UK" could all be used to identify the country which has the ISO country code `GB`. It is particularly implemented in [OpenRefine](https://openrefine.org/).

  The plugin adds a `/-/reconcile` endpoint to a table served by datasette, which responds based on the Reconciliation Service API specification. In order to activate this endpoint you need to configure the reconciliation service, as dscribed in the [usage](#usage) section.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-reconcile

  ## Usage

  ### Plugin configuration

  The plugin should be configured using Datasette's [`metadata.json`](https://docs.datasette.io/en/stable/metadata.html) file. The configuration can be put at the root, database or table layer of `metadata.json`, for most use cases it will make most sense to configure at the table level.

  Add a `datasette-reconcile` object under `plugins` in `metadata.json`. This should look something like:

  ```json
  {
    "databases": {
      "sf-trees": {
        "tables": {
          "Street_Tree_List": {
            "plugins": {
              "datasette-reconcile": {
                "id_field": "id",
                "name_field": "name",
                "type_field": "type",
                "type_default": [
                  {
                    "id": "tree",
                    "name": "Tree"
                  }
                ],
                "max_limit": 5,
                "service_name": "Tree reconciliation",
                "view_url": "https://example.com/trees/{{id}}"
              }
            }
          }
        }
      }
    }
  }
  ```

  The only required item in the configuration is `name_field`. This refers to the field in the table which will be searched to match the query text.

  The rest of the configuration items are optional, and are as follows:

  - `id_field`: The field containing the identifier for this entity. If not provided, and there is a primary key set, then the primary key will be used. A primary key of more than one field will give an error.
  - `type_field`: If provided, this field will be used to determine the type of the entity. If not provided, then the `type_default` setting will be used instead.
  - `type_default`: If provided, this value will be used as the type of every entity returned. If not provided the default of `Object` will be used for every entity.
  - `max_limit`: The maximum number of records that a query can request to return. This is 5 by default. A individual query can request fewer results than this, but it cannot request more.
  - `service_name`: The name of the reconciliation service that will appear in the service manifest. If not provided it will take the form `<database name> <table name> reconciliation`.
  - `identifierSpace`: [Identifier space](https://reconciliation-api.github.io/specs/latest/#identifier-and-schema-spaces) given in the service manifest. If not provided a default of `http://rdf.freebase.com/ns/type.object.id` is used.
  - `schemaSpace`: [Schema space](https://reconciliation-api.github.io/specs/latest/#identifier-and-schema-spaces) given in the service manifest. If not provided a default of `http://rdf.freebase.com/ns/type.object.id` is used.
  - `view_url`: [URL for a view of an individual entity](https://reconciliation-api.github.io/specs/latest/#dfn-view-template). It must contain the string `{{id}}` which will be replaced with the ID of the entity. If not provided it will use the default datasette view for the entity record (something like `/<db_name>/<table>/{{id}}`).

  ### Using the endpoint

  Once the plugin is configured for a particular database or table, you can access the reconciliation endpoint using the url `/<db_name>/<table>/-/reconcile`.

  A simple GET request to `/<db_name>/<table>/-/reconcile` will return the [Service Manifest](https://reconciliation-api.github.io/specs/latest/#service-manifest) as JSON which reconciliation clients can use to determine how the service is set up.

  A POST request to the same url with the `queries` argument set will trigger the reconciliation process. The `queries` parameter should be a json object in the format described in [the specification](https://reconciliation-api.github.io/specs/latest/#reconciliation-queries). An example set of two queries would look like:

  ```json
  {
    "q1": {
      "query": "Hans-Eberhard Urbaniak"
    },
    "q2": {
      "query": "Ernst Schwanhold"
    }
  }
  ```

  The query can optionally be encoded as a `queries` parameter in a GET request. For example:

  ```
  /<db_name>/<table>/-/reconcile?queries={"q1":{"query":"Hans-Eberhard Urbaniak"},"q2":{"query": "Ernst Schwanhold"}}
  ```

  Various options are available in the query object. Current the only ones implemented in datasette-reconcile are the mandatory `query` string, and the `limit` option, which must be less than or equal to the value in the `max_limit` configration option.

  All endpoints that start with `/<db_name>/<table>/-/reconcile` are configured to send an `Access-Control-Allow-Origin: *` CORS header to allow access [as described in the specification](https://reconciliation-api.github.io/specs/latest/#cross-origin-access).

  JSONP output is not yet supported.

  ### Returned value

  The result of the GET or POST `queries` requests described above is a json object describing potential [reconciliation candidates](https://reconciliation-api.github.io/specs/latest/#reconciliation-query-responses) for each of the queries specified. The result will look something like:

  ```json
  {
    "q1": {
      "result": [
        {
          "id": "120333937",
          "name": "Urbaniak, Regina",
          "score": 53.015232,
          "match": false,
          "type": [
            {
              "id": "person",
              "name": "Person"
            }
          ]
        },
        {
          "id": "1127147390",
          "name": "Urbaniak, Jan",
          "score": 52.357353,
          "match": false,
          "type": [
            {
              "id": "person",
              "name": "Person"
            }
          ]
        }
      ]
    },
    "q2": {
      "result": [
        {
          "id": "123064325",
          "name": "Schwanhold, Ernst",
          "score": 86.43497,
          "match": true,
          "type": [
            {
              "id": "person",
              "name": "Person"
            }
          ]
        },
        {
          "id": "116362988X",
          "name": "Schwanhold, Nadine",
          "score": 62.04763,
          "match": false,
          "type": [
            {
              "id": "person",
              "name": "Person"
            }
          ]
        }
      ]
    }
  }
  ```

  ### Behind the scenes

  The reconcile engine works by performing an SQL query against the `name_field` within the specified database table. Where that table has a full text search index implemented, the search will be performed against that index.

  When a full text search index is present on the table, the SQL query takes the form (based on the search query `test`, note that double quotes are added to facilitate searching - these are not present in the original query):

  ```sql
  select <id_field>, <name_field>
  from <table>
    inner join (
      select "rowid", "rank"
      from <fts_table>
      where <fts_table> MATCH '"test"'
    ) as "a" on <table>."rowid" = a."rowid"
  order by a.rank
  limit 5
  ```

  If a full text search index is not present, the query looks like this (note that the wildcard `%` is added to either side of the query - these are not present in the original query):

  ```sql
  select <id_field>, <name_field>
  from <table>
  where <name_field> like '%test%'
  limit 5
  ```

  ### Extend endpoint

  You can also use the reconciliation API [Data extension service](https://www.w3.org/community/reports/reconciliation/CG-FINAL-specs-0.2-20230410/#data-extension-service) to find additional properties for a set of entities, given an ID.

  Send a GET request to the `/<db_name>/<table>/-/reconcile/extend/propose` endpoint to find a list of the possible properties you can select. The properties are all the columns in the table (excluding any that have been hidden). An example response would look like:

  ```json
  {
    "limit": 5,
    "type": "Person",
    "properties": [
      {
        "id": "preferredName",
        "name": "preferredName"
      },
      {
        "id": "professionOrOccupation",
        "name": "professionOrOccupation"
      },
      {
        "id": "wikidataId",
        "name": "wikidataId"
      }
    ]
  }
  ```

  Then send a POST request to the `/<db_name>/<table>/-/reconcile` endpoint with an `extend` argument. The `extend` argument should be a JSON object with a set of `ids` to lookup and `properties` to return. For example:

  ```json
  {
    "ids": ["10662041X", "1064905412"],
    "properties": [
      {
        "id": "professionOrOccupation"
      },
      {
        "id": "wikidataId"
      }
    ]
  }
  ```

  The endpoint will return a result that looks like:

  ```json
  {
    "meta": [
      {
        "id": "professionOrOccupation",
        "name": "professionOrOccupation"
      },
      {
        "id": "wikidataId",
        "name": "wikidataId"
      }
    ],
    "rows": {
      "10662041X": {
        "professionOrOccupation": [
          {
            "str": "Doctor"
          }
        ],
        "wikidataId": [
          {
            "str": "Q3874347"
          }
        ]
      },
      "1064905412": {
        "professionOrOccupation": [
          {
            "str": "Architect"
          }
        ],
        "wikidataId": [
          {
            "str": "Q3874347"
          }
        ]
      }
    }
  }
  ```

  ### Suggest endpoints

  You can also use the [suggest endpoints](https://www.w3.org/community/reports/reconciliation/CG-FINAL-specs-0.2-20230410/#suggest-services) to get quick suggestions, for example for an auto-complete dropdown menu. The following endpoints are available:

  - `/<db_name>/<table>/-/reconcile/suggest/property` - looks up in a list of table columns
  - `/<db_name>/<table>/-/reconcile/suggest/entity` - looks up in a list of table rows
  - `/<db_name>/<table>/-/reconcile/suggest/type` - not currently implemented

  Each endpoint takes a `prefix` argument which can be used in a GET request. For example, the GET request `/<db_name>/<table>/-/reconcile/suggest/entity?prefix=abc` will produce a response such as:

  ```json
  {
    "result": [
      {
        "name": "abc company limited",
        "id": "Q123456"
      },
      {
        "name": "abc other company limited",
        "id": "Q123457"
      }
    ]
  }
  ```

  ## Development

  This plugin uses hatch for build and testing. To set up this plugin locally, first checkout the code.

  You'll need to fetch the git submodules for the tests too:

      git submodule init
      git submodule update

  To run the tests:

      hatch run test

  Run tests then report on coverage

      hatch run cov

  Run tests then run a server showing where coverage is missing

      hatch run cov-html

  ### Linting/formatting

  Black and ruff should be run before committing any changes.

  To check for any changes needed:

      hatch run lint:style

  To run any autoformatting possible:

      hatch run lint:fmt

  ### Publish to pypi

      hatch build
      hatch publish
      git tag v<VERSION_NUMBER>
      git push origin v<VERSION_NUMBER>

  ## Acknowledgements

  Thanks for [@simonw](https://github.com/simonw/) for developing datasette and the datasette ecosystem.

  Other contributions from:

  - [@JBPressac](https://github.com/JBPressac/)
  - [@nicokant](https://github.com/nicokant/) - implementation of extend service

project_urls:
  {"CI": "https://github.com/drkane/datasette-reconcile/actions", "Changelog": "https://github.com/drkane/datasette-reconcile/releases", "Documentation": "https://github.com/drkane/datasette-reconcile#readme", "Issues": "https://github.com/drkane/datasette-reconcile/issues", "Source": "https://github.com/drkane/datasette-reconcile"}

requires_dist:
  ["datasette", "fuzzywuzzy[speedup]", "mypy>=1.0.0; extra == 'lint'", "ruff>=0.1.8; extra == 'lint'", "coverage[toml]>=6.5; extra == 'test'", "httpx; extra == 'test'", "jsonschema; extra == 'test'", "pytest; extra == 'test'", "pytest-asyncio; extra == 'test'", "sqlite-utils; extra == 'test'"]


name: datasette-redirect-forbidden
summary: Redirect forbidden requests to a login page
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-redirect-forbidden
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-redirect-forbidden/
project_url: https://pypi.org/project/datasette-redirect-forbidden/
release_url: https://pypi.org/project/datasette-redirect-forbidden/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1
description:
  # datasette-redirect-forbidden

  [![PyPI](https://img.shields.io/pypi/v/datasette-redirect-forbidden.svg)](https://pypi.org/project/datasette-redirect-forbidden/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-redirect-forbidden?include_prereleases&label=changelog)](https://github.com/simonw/datasette-redirect-forbidden/releases)
  [![Tests](https://github.com/simonw/datasette-redirect-forbidden/workflows/Test/badge.svg)](https://github.com/simonw/datasette-redirect-forbidden/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-redirect-forbidden/blob/main/LICENSE)

  Redirect forbidden requests to a login page

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-redirect-forbidden

  ## Usage

  Add the following to your `metadata.yml` (or `metadata.json`) file to configure the plugin:

  ```yaml
  plugins:
    datasette-redirect-forbidden:
      redirect_to: /-/login
  ```
  Any 403 forbidden pages will redirect to the specified page.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-redirect-forbidden
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-redirect-forbidden/actions", "Changelog": "https://github.com/simonw/datasette-redirect-forbidden/releases", "Homepage": "https://github.com/simonw/datasette-redirect-forbidden", "Issues": "https://github.com/simonw/datasette-redirect-forbidden/issues"}


name: datasette-redirect-to-https
summary: Datasette plugin that redirects all non-https requests to https
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-redirect-to-https
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-redirect-to-https/
project_url: https://pypi.org/project/datasette-redirect-to-https/
release_url: https://pypi.org/project/datasette-redirect-to-https/0.2/
requires_python: >=3.6
version: 0.2
description:
  # datasette-redirect-to-https

  [![PyPI](https://img.shields.io/pypi/v/datasette-redirect-to-https.svg)](https://pypi.org/project/datasette-redirect-to-https/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-redirect-to-https?include_prereleases&label=changelog)](https://github.com/simonw/datasette-redirect-to-https/releases)
  [![Tests](https://github.com/simonw/datasette-redirect-to-https/workflows/Test/badge.svg)](https://github.com/simonw/datasette-redirect-to-https/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-redirect-to-https/blob/main/LICENSE)

  Datasette plugin that redirects all non-https requests to https

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-redirect-to-https

  ## Usage

  Once installed, incoming GET requests to the `http://` protocol will be 301 redirected to the `https://` equivalent page.

  HTTP verbs other than GET will get a 405 Method Not Allowed HTTP error.

  ## Configuration

  Some hosting providers handle HTTPS for you, passing requests back to your application server over HTTP.

  For this plugin to work correctly, you need to detect that they the original incoming request came in over HTTP.

  Hosting providers like this often set an additional HTTP header such as `x-forwarded-proto: http` to let you know the original protocol.

  You can configure `datasette-redirect-to-https` to respect this header using the following plugin configuration in `metadata.json`:

  ```json
  {
    "plugins": {
      "datasette-redirect-to-https": {
        "if_headers": {
          "x-forwarded-proto": "http"
        }
      }
    }
  }
  ```
  The above example will redirect to `https://` if the incoming request has a `x-forwarded-proto: http` request header.

  If multiple `if_headers` are listed, the redirect will occur if any of them match.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-redirect-to-https
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-redirect-to-https/actions", "Changelog": "https://github.com/simonw/datasette-redirect-to-https/releases", "Homepage": "https://github.com/simonw/datasette-redirect-to-https", "Issues": "https://github.com/simonw/datasette-redirect-to-https/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgi-lifespan ; extra == 'test'"]


name: datasette-redirects
summary: Configure redirects for a Datasette instance
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-redirects/
project_url: https://pypi.org/project/datasette-redirects/
release_url: https://pypi.org/project/datasette-redirects/0.1/
requires_dist: ["datasette"]
requires_python: >=3.10
version: 0.1
description:
  # datasette-redirects

  [![PyPI](https://img.shields.io/pypi/v/datasette-redirects.svg)](https://pypi.org/project/datasette-redirects/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-redirects?include_prereleases&label=changelog)](https://github.com/datasette/datasette-redirects/releases)
  [![Tests](https://github.com/datasette/datasette-redirects/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-redirects/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-redirects/blob/main/LICENSE)

  Configure redirects for a Datasette instance

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-redirects
  ```
  ## Usage

  Configure redirects in your `datasette.yaml` configuration file. Redirects are organized by HTTP status code:

  ```yaml
  plugins:
    datasette-redirects:
      301:
        old-page: https://example.com/new-page
        another-old-page: /new-location
      302:
        temporary-page: /temporary-location
  ```

  This configuration will:

  - Redirect `/old-page` to `https://example.com/new-page` with a 301 (permanent) status
  - Redirect `/another-old-page` to `/new-location` with a 301 status
  - Redirect `/temporary-page` to `/temporary-location` with a 302 (temporary) status

  Both `/path` and `/path/` variants are matched automatically - you don't need to configure both.

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-redirects
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-redirects/actions", "Changelog": "https://github.com/datasette/datasette-redirects/releases", "Homepage": "https://github.com/datasette/datasette-redirects", "Issues": "https://github.com/datasette/datasette-redirects/issues"}


name: datasette-referrer-policy
summary: Set the Referrer-Policy header for a Datasette site
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-referrer-policy/
project_url: https://pypi.org/project/datasette-referrer-policy/
release_url: https://pypi.org/project/datasette-referrer-policy/0.1/
requires_dist: ["datasette"]
requires_python: >=3.10
version: 0.1
description:
  # datasette-referrer-policy

  [![PyPI](https://img.shields.io/pypi/v/datasette-referrer-policy.svg)](https://pypi.org/project/datasette-referrer-policy/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-referrer-policy?include_prereleases&label=changelog)](https://github.com/datasette/datasette-referrer-policy/releases)
  [![Tests](https://github.com/datasette/datasette-referrer-policy/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-referrer-policy/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-referrer-policy/blob/main/LICENSE)

  Set the Referrer-Policy header for a Datasette site

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-referrer-policy
  ```
  ## Usage

  Installing this plugin causes every page on your Datasette instance to return this header:

      Referrer-Policy: strict-origin-when-cross-origin

  You can configure a different policy using this configuration:

  ```yaml
  plugins:
    datasette-referrer-policy:
      policy: no-referrer
  ```
  See [MDN: Referrer-Policy header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Referrer-Policy) for other values.

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-referrer-policy
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-referrer-policy/actions", "Changelog": "https://github.com/datasette/datasette-referrer-policy/releases", "Homepage": "https://github.com/datasette/datasette-referrer-policy", "Issues": "https://github.com/datasette/datasette-referrer-policy/issues"}


name: datasette-remote-actors
summary: Datasette plugin for fetching details of actors from a remote endpoint
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-remote-actors/
project_url: https://pypi.org/project/datasette-remote-actors/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-remote-actors/0.1a7/
version: 0.1a7
description:
  # datasette-remote-actors

  [![PyPI](https://img.shields.io/pypi/v/datasette-remote-actors.svg)](https://pypi.org/project/datasette-remote-actors/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-remote-actors?include_prereleases&label=changelog)](https://github.com/datasette/datasette-remote-actors/releases)
  [![Test](https://github.com/datasette/datasette-remote-actors/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-remote-actors/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-remote-actors/blob/main/LICENSE)

  A Datasette plugin for fetching details of actors (users or API clients) from a remote, centralized JSON endpoint.

  See [Datasette issue #2180](https://github.com/simonw/datasette/issues/2180) for background on this plugin.

  ## Installation

  ```bash
  datasette install datasette-remote-actors
  ```

  ## API Endpoint Requirements

  Configure this plugin with a URL pointing to an API endpoint that provides actor information.

  This endpoint **must**:

  1. Accept a GET request.
  2. Accept a query string parameter `ids` containing a comma-separated list of actor IDs (e.g., `?ids=1,a2,b3`).
  3. Return a JSON response with a `200 OK` status code upon success.
  4. The JSON response body **must** be a dictionary where the keys are the **string representations** of the actor IDs requested, and the values are dictionaries containing the details for each actor.

  **Example Request:**

  `GET /path/to/your/endpoint?ids=1,2`

  **Example Successful JSON Response:**

  ```json
  {
    "1": {
      "id": "1",
      "name": "Cleopatra",
      "email": "cleo@example.com"
    },
    "2": {
      "id": 2,
      "name": "Julius Caesar",
      "username": "jcaesar"
    }
  }
  ```

  **Important Notes:**

  * The **keys** in the returned JSON object (`"1"`, `"2"` in the example) **must be strings**, even if the original IDs are integers.
  * Each actor dictionary value **must** contain an `id` field, which can be a string or an integer matching the requested ID. Other fields (like `name`, `username`, `email`) are flexible and defined by your endpoint.
  * **Small Actor Sets:** If you have a small, fixed number of actors, your endpoint can optionally ignore the `?ids=` parameter and always return the *entire* dictionary of all known actors. The plugin will cache these and use the cached data for subsequent lookups (if `ttl` is configured).
  * **Error Handling:** If the remote endpoint returns a non-200 status code, times out, or returns invalid JSON, this plugin will log an error (if Datasette logging is configured) and will not return actor data for the requested IDs for that request.

  ## Configuration

  You configure the plugin using Datasette's `metadata.json` or `metadata.yaml` file.

  **Minimal Configuration:**

  ```yaml
  plugins:
    datasette-remote-actors:
      url: https://example.com/actors.json
  ```

  **Full Configuration Example:**

  ```yaml
  plugins:
    datasette-remote-actors:
      # (Required) URL to the remote actor endpoint
      url: https://example.com/actors.json

      # (Optional) Cache Time-To-Live in seconds.
      # If set, actor details will be cached in memory.
      # Uses cachetools.TTLCache with a default maxsize of 1000.
      # Omit for no caching.
      ttl: 60

      # (Optional) Bearer token for authentication.
      # If set, adds an "Authorization: Bearer <token>" header to the request.
      token: your-secret-api-token

      # (Optional) Enable integration with datasette-profiles.
      # Requires datasette-profiles plugin to be installed.
      # See section below for details.
      datasette-profiles: true
  ```

  **Configuration Options:**

  * `url` (string, **required**): The URL to the endpoint that can resolve actor IDs into JSON actor dictionaries.
  * `ttl` (integer, optional): The number of seconds to cache the result for a specific actor ID. Uses an in-memory `TTLCache` (default `maxsize=1000`). Omit this or set to `0` for no caching.
  * `token` (string, optional): An authentication token. If provided, it will be sent in the `Authorization: Bearer <token>` HTTP header when calling the `url`.
  * `datasette-profiles` (boolean, optional): Set to `true` to enable integration with the [datasette-profiles](https://github.com/datasette/datasette-profiles) plugin. Defaults to `false`.

  ## Integration with datasette-profiles

  If you want to allow users to override or supplement the actor details fetched from the remote endpoint with their own profile information stored within Datasette, you can use the [datasette-profiles](https://github.com/datasette/datasette-profiles) plugin.

  1.  Install the companion plugin:
      ```bash
      datasette install datasette-profiles
      ```
  2.  Enable the integration in the `datasette-remote-actors` configuration:
      ```yaml
      plugins:
        datasette-remote-actors:
          url: https://example.com/actors.json
          datasette-profiles: true
          # other options...
      ```

  When enabled:

  * The plugin will first fetch actor details from the configured `url`.
  * It will then query the `profiles` table (created by `datasette-profiles`) in Datasette's internal database for matching actor IDs.
  * If a profile exists for an actor ID, the data from the `profiles` table will be merged into the actor dictionary fetched from the remote URL. Any non-null values in the `profiles` table (except the `id` column itself) will overwrite corresponding keys from the remote data.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

  ```bash
  cd datasette-remote-actors
  python3 -m venv venv
  source venv/bin/activate
  ```

  Now install the dependencies and test dependencies:

  ```bash
  pip install -e '.[test]'
  ```

  To run the tests:

  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-remote-actors/actions", "Changelog": "https://github.com/datasette/datasette-remote-actors/releases", "Homepage": "https://github.com/datasette/datasette-remote-actors", "Issues": "https://github.com/datasette/datasette-remote-actors/issues"}

requires_dist:
  ["datasette>=1.0a21", "cachetools", "httpx", "pytest; extra == \"test\"", "pytest-httpx; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "datasette-profiles>=0.1a3; extra == \"test\""]


name: datasette-remote-metadata
summary: Periodically refresh Datasette metadata from a remote URL
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-remote-metadata
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-remote-metadata/
project_url: https://pypi.org/project/datasette-remote-metadata/
release_url: https://pypi.org/project/datasette-remote-metadata/0.1/
requires_python: >=3.6
version: 0.1
description:
  # datasette-remote-metadata

  [![PyPI](https://img.shields.io/pypi/v/datasette-remote-metadata.svg)](https://pypi.org/project/datasette-remote-metadata/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-remote-metadata?include_prereleases&label=changelog)](https://github.com/simonw/datasette-remote-metadata/releases)
  [![Tests](https://github.com/simonw/datasette-remote-metadata/workflows/Test/badge.svg)](https://github.com/simonw/datasette-remote-metadata/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-remote-metadata/blob/main/LICENSE)

  Periodically refresh Datasette metadata from a remote URL

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-remote-metadata

  ## Usage

  Add the following to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-remote-metadata": {
              "url": "https://example.com/remote-metadata.yml"
          }
      }
  }
  ```
  The plugin will fetch the specified metadata  from that URL at startup and combine it with any existing metadata. You can use a URL to either a JSON file or a YAML file.

  It will periodically refresh that metadata - by default every 30 seconds, unless you specify an alternative `"ttl"` value in the plugin configuration.

  ## Configuration

  Available configuration options are as follows:

  - `"url"` - the URL to retrieve remote metadata from. Can link to a JSON or a YAML file.
  - `"ttl"` - integer value in secords: how frequently should the script check for fresh metadata. Defaults to 30 seconds.
  - `"headers"` - a dictionary of additional request headers to send.
  - `"cachebust"` - if true, a random `?0.29508` value will be added to the query string of the remote metadata to bust any intermediary caches.

  This example `metadata.json` configuration refreshes every 10 seconds, uses cache busting and sends an `Authorization: Bearer xyz` header with the request:

  ```json
  {
      "plugins": {
          "datasette-remote-metadata": {
              "url": "https://example.com/remote-metadata.yml",
              "ttl": 10,
              "cachebust": true,
              "headers": {
                  "Authorization": "Bearer xyz"
              }
          }
      }
  }
  ```
  This example if you are using `metadata.yaml` for configuration:
  ```yaml
  plugins:
    datasette-remote-metadata:
      url: https://example.com/remote-metadata.yml
      ttl: 10
      cachebust: true
      headers:
        Authorization: Bearer xyz
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-remote-metadata
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-remote-metadata/actions", "Changelog": "https://github.com/simonw/datasette-remote-metadata/releases", "Homepage": "https://github.com/simonw/datasette-remote-metadata", "Issues": "https://github.com/simonw/datasette-remote-metadata/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-httpx ; extra == 'test'"]


name: datasette-remove-database
summary: Remove a database from Datasette
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
license_expression: Apache-2.0
package_url: https://pypi.org/project/datasette-remove-database/
project_url: https://pypi.org/project/datasette-remove-database/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-remove-database/0.1a2/
requires_dist: ["datasette>=1.0a21", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]
requires_python: >=3.10
version: 0.1a2
description:
  # datasette-remove-database

  [![PyPI](https://img.shields.io/pypi/v/datasette-remove-database.svg)](https://pypi.org/project/datasette-remove-database/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-remove-database?include_prereleases&label=changelog)](https://github.com/datasette/datasette-remove-database/releases)
  [![Tests](https://github.com/datasette/datasette-remove-database/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-remove-database/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-remove-database/blob/main/LICENSE)

  Remove a database from Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-remove-database
  ```
  ## Usage

  Users with the `remove-database` permission will see a new menu item on the database page that allows them to remove the database.

  By default databases that are removed from Datasette will still be available on disk.

  To also delete the file from disk, add this configuration:
  ```yaml
  plugins:
    datasette-remove-database:
      delete: true
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-remove-database
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  python -m pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-remove-database/actions", "Changelog": "https://github.com/datasette/datasette-remove-database/releases", "Homepage": "https://github.com/datasette/datasette-remove-database", "Issues": "https://github.com/datasette/datasette-remove-database/issues"}


name: datasette-render-binary
summary: Datasette plugin for rendering binary data
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/plugins/datasette-render-binary
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-render-binary/
project_url: https://pypi.org/project/datasette-render-binary/
release_url: https://pypi.org/project/datasette-render-binary/0.3.1/
requires_dist: ["datasette", "filetype", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.3.1
description:
  # datasette-render-binary

  [![PyPI](https://img.shields.io/pypi/v/datasette-render-binary.svg)](https://pypi.org/project/datasette-render-binary/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-render-binary?include_prereleases&label=changelog)](https://github.com/simonw/datasette-render-binary/releases)
  [![Tests](https://github.com/simonw/datasette-render-binary/workflows/Test/badge.svg)](https://github.com/simonw/datasette-render-binary/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-render-binary/blob/main/LICENSE)

  Datasette plugin for rendering binary data.

  Install this plugin in the same environment as Datasette to enable this new functionality:

      pip install datasette-render-binary

  Binary data in cells will now be rendered as a mixture of characters and octets.

  ![Screenshot](https://raw.githubusercontent.com/simonw/datasette-render-binary/master/example.png)

project_urls:
  {"CI": "https://github.com/simonw/datasette-render-binary/actions", "Changelog": "https://github.com/simonw/datasette-render-binary/releases", "Homepage": "https://datasette.io/plugins/datasette-render-binary", "Issues": "https://github.com/simonw/datasette-render-binary/issues"}


name: datasette-render-image-tags
summary: Turn any URLs ending in .jpg/.png/.gif into img tags with width 200
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-render-image-tags
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-render-image-tags/
project_url: https://pypi.org/project/datasette-render-image-tags/
release_url: https://pypi.org/project/datasette-render-image-tags/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-render-image-tags

  [![PyPI](https://img.shields.io/pypi/v/datasette-render-image-tags.svg)](https://pypi.org/project/datasette-render-image-tags/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-render-image-tags?include_prereleases&label=changelog)](https://github.com/simonw/datasette-render-image-tags/releases)
  [![Tests](https://github.com/simonw/datasette-render-image-tags/workflows/Test/badge.svg)](https://github.com/simonw/datasette-render-image-tags/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-render-image-tags/blob/main/LICENSE)

  Turn any URLs ending in .jpg/.png/.gif into img tags with width 200

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-render-image-tags

  ## Usage

  Once installed, any cells contaning a URL that ends with `.png` or `.jpg` or `.jpeg` or `.gif` will be rendered using an image tag, with a width of 200px.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-render-image-tags
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-render-image-tags/actions", "Changelog": "https://github.com/simonw/datasette-render-image-tags/releases", "Homepage": "https://github.com/simonw/datasette-render-image-tags", "Issues": "https://github.com/simonw/datasette-render-image-tags/issues"}


name: datasette-render-timestamps
summary: Datasette plugin for rendering timestamps
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-render-timestamps
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-render-timestamps/
project_url: https://pypi.org/project/datasette-render-timestamps/
project_urls: {"Homepage": "https://github.com/simonw/datasette-render-timestamps"}
release_url: https://pypi.org/project/datasette-render-timestamps/1.0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'"]
version: 1.0.1
description:
  # datasette-render-timestamps

  [![PyPI](https://img.shields.io/pypi/v/datasette-render-timestamps.svg)](https://pypi.org/project/datasette-render-timestamps/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-render-timestamps.svg?style=svg)](https://circleci.com/gh/simonw/datasette-render-timestamps)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-render-timestamps/blob/master/LICENSE)

  Datasette plugin for rendering timestamps.

  ## Installation

  Install this plugin in the same environment as Datasette to enable this new functionality:

      pip install datasette-render-timestamps

  The plugin will then look out for integer numbers that are likely to be timestamps - anything that would be a number of seconds from 5 years ago to 5 years in the future.

  These will then be rendered in a more readable format.

  ## Configuration

  You can disable automatic column detection in favour of explicitly listing the columns that you would like to render using [plugin configuration](https://datasette.readthedocs.io/en/stable/plugins.html#plugin-configuration) in a `metadata.json` file.

  Add a `"datasette-render-timestamps"` configuration block and use a `"columns"` key to list the columns you would like to treat as timestamp values:

  ```json
  {
      "plugins": {
          "datasette-render-timestamps": {
              "columns": ["created", "updated"]
          }
      }
  }
  ```
  This will cause any `created` or `updated` columns in any table to be treated as timestamps and rendered.

  Save this to `metadata.json` and run datasette with the `--metadata` flag to load this configuration:

      datasette serve mydata.db --metadata metadata.json

  To disable automatic timestamp detection entirely, you can use `"columnns": []`.

  This configuration block can be used at the top level, or it can be applied just to specific databases or tables. Here's how to apply it to just the `entries` table in the `news.db` database:

  ```json
  {
      "databases": {
          "news": {
              "tables": {
                  "entries": {
                      "plugins": {
                          "datasette-render-timestamps": {
                              "columns": ["created", "updated"]
                          }
                      }
                  }
              }
          }
      }
  }
  ```

  ### Customizing the date format

  The default format is `%B %d, %Y - %H:%M:%S UTC` which renders for example: `October 10, 2019 - 07:18:29 UTC`. If you want another format, the date format can be customized using plugin configuration. Any format string supported by [strftime](http://strftime.org/) may be used. For example:

  ```json
  {
      "plugins": {
          "datasette-render-timestamps": {
              "format": "%Y-%m-%d-%H:%M:%S"
          }
      }
  }
  ```


name: datasette-ripgrep
summary: Web interface for searching your code using ripgrep, built as a Datasette plugin
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-ripgrep
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-ripgrep/
project_url: https://pypi.org/project/datasette-ripgrep/
release_url: https://pypi.org/project/datasette-ripgrep/0.8.2/
requires_python: >=3.7
version: 0.8.2
description:
  # datasette-ripgrep

  [![PyPI](https://img.shields.io/pypi/v/datasette-ripgrep.svg)](https://pypi.org/project/datasette-ripgrep/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-ripgrep?include_prereleases&label=changelog)](https://github.com/simonw/datasette-ripgrep/releases)
  [![Tests](https://github.com/simonw/datasette-ripgrep/workflows/Test/badge.svg)](https://github.com/simonw/datasette-ripgrep/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-ripgrep/blob/main/LICENSE)

  Web interface for searching your code using [ripgrep](https://github.com/BurntSushi/ripgrep), built as a [Datasette](https://datasette.io/) plugin

  For background on this project see [datasette-ripgrep: deploy a regular expression search engine for your source code](https://simonwillison.net/2020/Nov/28/datasette-ripgrep/).

  ## Demo

  Try this plugin out at https://ripgrep.datasette.io/-/ripgrep - where you can run regular expression searches across the source code of Datasette and all of the `datasette-*` plugins belonging to the [simonw GitHub user](https://github.com/simonw).

  Some example searches:

  - [with.\*AsyncClient](https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient) - regular expression search for `with.*AsyncClient`
  - [.plugin_config, literal=on](https://ripgrep.datasette.io/-/ripgrep?pattern=.plugin_config\(&literal=on) - a non-regular expression search for `.plugin_config(`
  - [with.\*AsyncClient glob=datasette/\*\*](https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient&glob=datasette%2F%2A%2A) - search for that pattern only within the `datasette/` top folder
  - ["sqlite-utils\[">\] glob=setup.py](https://ripgrep.datasette.io/-/ripgrep?pattern=%22sqlite-utils%5B%22%3E%5D&glob=setup.py) - a regular expression search for packages that depend on either `sqlite-utils` or `sqlite-utils>=some-version`
  - [test glob=!\*.html](https://ripgrep.datasette.io/-/ripgrep?pattern=test&glob=%21*.html) - search for the string `test` but exclude results in HTML files

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-ripgrep

  The `rg` executable needs to be [installed](https://github.com/BurntSushi/ripgrep/blob/master/README.md#installation) such that it can be run by this tool.

  ## Usage

  This plugin requires configuration: it needs to a `path` setting so that it knows where to run searches.

  Create a `metadata.json` file that looks like this:

  ```json
  {
      "plugins": {
          "datasette-ripgrep": {
              "path": "/path/to/your/files"
          }
      }
  }
  ```

  Now run Datasette using `datasette -m metadata.json`. The plugin will add an interface at `/-/ripgrep` for running searches.

  ## Plugin configuration

  The `"path"` configuration is required. Optional extra configuration options are:

  - `time_limit` - floating point number. The `rg` process will be terminated if it takes longer than this limit. The default is one second, `1.0`.
  - `max_lines` - integer. The `rg` process will be terminated if it returns more than this number of lines. The default is `2000`.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-ripgrep
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-ripgrep/actions", "Changelog": "https://github.com/simonw/datasette-ripgrep/releases", "Homepage": "https://github.com/simonw/datasette-ripgrep", "Issues": "https://github.com/simonw/datasette-ripgrep/issues"}

requires_dist:
[cell repeats an earlier one in this file: 104 characters]


name: datasette-rure
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-rure
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-rure/
project_url: https://pypi.org/project/datasette-rure/
project_urls: {"Homepage": "https://github.com/simonw/datasette-rure"}
release_url: https://pypi.org/project/datasette-rure/0.3/
requires_dist: ["datasette", "rure", "pytest ; extra == 'test'"]
version: 0.3
summary:
  Datasette plugin that adds a custom SQL function for executing matches using the Rust regular expression engine

description:
  # datasette-rure

  [![PyPI](https://img.shields.io/pypi/v/datasette-rure.svg)](https://pypi.org/project/datasette-rure/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-rure.svg?style=svg)](https://circleci.com/gh/simonw/datasette-rure)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-rure/blob/master/LICENSE)

  Datasette plugin that adds a custom SQL function for executing matches using the Rust regular expression engine

  Install this plugin in the same environment as Datasette to enable the `regexp()` SQL function.

      $ pip install datasette-rure

  The plugin is built on top of the [rure-python](https://github.com/davidblewett/rure-python) library by David Blewett.

  ## regexp() to test regular expressions

  You can test if a value matches a regular expression like this:

      select regexp('hi.*there', 'hi there')
      -- returns 1
      select regexp('not.*there', 'hi there')
      -- returns 0

  You can also use SQLite's custom syntax to run matches:

      select 'hi there' REGEXP 'hi.*there'
      -- returns 1

  This means you can select rows based on regular expression matches - for example, to select every article where the title begins with an E or an F:

      select * from articles where title REGEXP '^[EF]'

  Try this out: [REGEXP interactive demo](https://datasette-rure-demo.datasette.io/24ways?sql=select+*+from+articles+where+title+REGEXP+%27%5E%5BEF%5D%27)

  ## regexp_match() to extract groups

  You can extract captured subsets of a pattern using `regexp_match()`.

      select regexp_match('.*( and .*)', title) as n from articles where n is not null
      -- Returns the ' and X' component of any matching titles, e.g.
      --     and Recognition
      --     and Transitions Their Place
      -- etc

  This will return the first parenthesis match when called with two arguments. You can call it with three arguments to indicate which match you would like to extract:

      select regexp_match('.*(and)(.*)', title, 2) as n from articles where n is not null

  The function will return `null` for invalid inputs e.g. a pattern without capture groups.

  Try this out: [regexp_match() interactive demo](https://datasette-rure-demo.datasette.io/24ways?sql=select+%27WHY+%27+%7C%7C+regexp_match%28%27Why+%28.*%29%27%2C+title%29+as+t+from+articles+where+t+is+not+null)

  ## regexp_matches() to extract multiple matches at once

  The `regexp_matches()` function can be used to extract multiple patterns from a single string. The result is returned as a JSON array, which can then be further processed using SQLite's [JSON functions](https://www.sqlite.org/json1.html).

  The first argument is a regular expression with named capture groups. The second argument is the string to be matched.

      select regexp_matches(
          'hello (?P<name>\w+) the (?P<species>\w+)',
          'hello bob the dog, hello maggie the cat, hello tarquin the otter'
      )

  This will return a list of JSON objects, each one representing the named captures from the original regular expression:

      [
          {"name": "bob", "species": "dog"},
          {"name": "maggie", "species": "cat"},
          {"name": "tarquin", "species": "otter"}
      ]

  Try this out: [regexp_matches() interactive demo](https://datasette-rure-demo.datasette.io/24ways?sql=select+regexp_matches%28%0D%0A++++%27hello+%28%3FP%3Cname%3E%5Cw%2B%29+the+%28%3FP%3Cspecies%3E%5Cw%2B%29%27%2C%0D%0A++++%27hello+bob+the+dog%2C+hello+maggie+the+cat%2C+hello+tarquin+the+otter%27%0D%0A%29)


name: datasette-sandstorm-support
summary: Authentication and permissions for Datasette on Sandstorm
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-sandstorm-support
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sandstorm-support/
project_url: https://pypi.org/project/datasette-sandstorm-support/
release_url: https://pypi.org/project/datasette-sandstorm-support/0.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.2
description:
  # datasette-sandstorm-support

  [![PyPI](https://img.shields.io/pypi/v/datasette-sandstorm-support.svg)](https://pypi.org/project/datasette-sandstorm-support/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-sandstorm-support?include_prereleases&label=changelog)](https://github.com/simonw/datasette-sandstorm-support/releases)
  [![Tests](https://github.com/simonw/datasette-sandstorm-support/workflows/Test/badge.svg)](https://github.com/simonw/datasette-sandstorm-support/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-sandstorm-support/blob/main/LICENSE)

  Authentication and permissions for Datasette on Sandstorm

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-sandstorm-support

  ## Usage

  This plugin is part of [datasette-sandstorm](https://github.com/ocdtrekkie/datasette-sandstorm).

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-sandstorm-support
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-sandstorm-support/actions", "Changelog": "https://github.com/simonw/datasette-sandstorm-support/releases", "Homepage": "https://github.com/simonw/datasette-sandstorm-support", "Issues": "https://github.com/simonw/datasette-sandstorm-support/issues"}


name: datasette-saved-queries
summary: Datasette plugin that lets users save and execute queries
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-saved-queries
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-saved-queries/
project_url: https://pypi.org/project/datasette-saved-queries/
release_url: https://pypi.org/project/datasette-saved-queries/0.2.2/
version: 0.2.2
description:
  # datasette-saved-queries

  [![PyPI](https://img.shields.io/pypi/v/datasette-saved-queries.svg)](https://pypi.org/project/datasette-saved-queries/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-saved-queries?label=changelog)](https://github.com/simonw/datasette-saved-queries/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-saved-queries/blob/master/LICENSE)

  Datasette plugin that lets users save and execute queries

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-saved-queries

  ## Usage

  When the plugin is installed Datasette will automatically create a `saved_queries` table in the first connected database when it starts up.

  It also creates a `save_query` writable canned query which you can use to save new queries.

  Queries that you save will be added to the query list on the database page.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-saved-queries
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-saved-queries/actions", "Changelog": "https://github.com/simonw/datasette-saved-queries/releases", "Homepage": "https://github.com/simonw/datasette-saved-queries", "Issues": "https://github.com/simonw/datasette-saved-queries/issues"}

requires_dist:
  ["datasette >=0.45", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-scale-to-zero
summary: Quit Datasette if it has not received traffic for a specified time period
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
license_expression: Apache-2.0
package_url: https://pypi.org/project/datasette-scale-to-zero/
project_url: https://pypi.org/project/datasette-scale-to-zero/
release_url: https://pypi.org/project/datasette-scale-to-zero/0.3.2/
requires_dist: ["datasette"]
requires_python: >=3.10
version: 0.3.2
description:
  # datasette-scale-to-zero

  [![PyPI](https://img.shields.io/pypi/v/datasette-scale-to-zero.svg)](https://pypi.org/project/datasette-scale-to-zero/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-scale-to-zero?include_prereleases&label=changelog)](https://github.com/simonw/datasette-scale-to-zero/releases)
  [![Tests](https://github.com/simonw/datasette-scale-to-zero/workflows/Test/badge.svg)](https://github.com/simonw/datasette-scale-to-zero/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-scale-to-zero/blob/main/LICENSE)

  Quit Datasette if it has not received traffic for a specified time period

  Some hosting providers such as [Fly](https://fly.io/) offer a scale to zero mechanism, where servers can shut down and will be automatically started when new traffic arrives.

  This plugin can be used to configure Datasette to quit X minutes (or seconds, or hours) after the last request it received. It can also cause the Datasette server to exit after a configured maximum time whether or not it is receiving traffic.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-scale-to-zero

  ## Configuration

  This plugin will only take effect if it has been configured with a `duration` or `max_age` value. Without at least one of these settings the plugin will do nothing.

  Add the following to your ``metadata.json`` or ``metadata.yml`` configuration file:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "duration": "10m"
          }
      }
  }
  ```
  This will cause Datasette to quit if it has not received any HTTP traffic for 10 minutes.

  You can set this value using a suffix of `m` for minutes, `h` for hours or `s` for seconds.

  To cause Datasette to exit if the server has been running for longer than a specific time, use `"max_age"`:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "max_age": "10h"
          }
      }
  }
  ```
  This example will exit the Datasette server if it has been running for more than ten hours.

  You can use `"duration"` and `"max_age"` together in the same configuration file:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "max_age": "10h",
              "duration": "5m"
          }
      }
  }
  ```
  This example will quit if no traffic has been received in five minutes, or if the server has been running for ten hours.

  ## Configuring a shutdown HTTP message

  You can also configure the plugin to send an HTTP request somewhere right before it quits, using the `"shutdown_url"` option:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "duration": "10m",
              "shutdown_url": "https://example.com/shutdown"
          }
      }
  }
  ```
  You can add additional headers to the GET request - for example to send Authorization headers - using `"shutdown_headers"`:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "duration": "10m",
              "shutdown_url": "https://example.com/shutdown",
              "shutdown_headers": {
                  "Authorization": "Bearer secret"
              }
          }
      }
  }
  ```
  Use `"shutdown_method"` to set a different HTTP method, e.g. for `POST`. You can also set `shutdown_body` to specify the body that should be sent with the request:

  ```json
  {
      "plugins": {
          "datasette-scale-to-zero": {
              "duration": "10m",
              "shutdown_url": "https://example.com/shutdown",
              "shutdown_method": "POST",
              "shutdown_headers": {
                  "Authorization": "Bearer secret",
                  "Content-Type": "application/json"
              },
              "shutdown_body": "{\"message\": \"shutting down\"}"
          }
      }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-scale-to-zero
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-scale-to-zero/actions", "Changelog": "https://github.com/simonw/datasette-scale-to-zero/releases", "Homepage": "https://github.com/simonw/datasette-scale-to-zero", "Issues": "https://github.com/simonw/datasette-scale-to-zero/issues"}


name: datasette-scan
summary: Datasette plugin for scanning for SQLite files
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-scan/
project_url: https://pypi.org/project/datasette-scan/
release_url: https://pypi.org/project/datasette-scan/0.1.1/
requires_dist: ["datasette", "sqlite-scanner>=0.1.1"]
requires_python: >=3.10
version: 0.1.1
description:
  # datasette-scan

  [![PyPI](https://img.shields.io/pypi/v/datasette-scan.svg)](https://pypi.org/project/datasette-scan/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-scan?include_prereleases&label=changelog)](https://github.com/simonw/datasette-scan/releases)
  [![Tests](https://github.com/simonw/datasette-scan/actions/workflows/test.yml/badge.svg)](https://github.com/simonw/datasette-scan/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-scan/blob/main/LICENSE)

  Datasette plugin that scans directories for SQLite database files and serves them.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-scan
  ```

  ## Usage

  Scan the current directory for SQLite files and serve them:

  ```bash
  datasette scan
  ```

  Scan a specific directory:

  ```bash
  datasette scan /path/to/directory
  ```

  Scan multiple directories and include explicit database files:

  ```bash
  datasette scan /data/projects /data/archives extra.db
  ```

  Specify a port (or any other `datasette serve` option):

  ```bash
  datasette scan -p 8001
  ```

  All options accepted by `datasette serve` are also accepted by `datasette scan` -- these are introspected at runtime, so new options added to future versions of Datasette will be picked up automatically.

  ### How it works

  - Positional arguments that are **directories** are scanned recursively for SQLite files using the [sqlite-scanner](https://pypi.org/project/sqlite-scanner/) binary, which identifies SQLite files by their magic bytes rather than file extension.
  - Positional arguments that are **files** are passed through directly as database files.
  - If no positional arguments are given, the current directory (`.`) is scanned.
  - Discovered files are then served as if you had run `datasette serve file1.db file2.db ...`.

  ### Continuous scanning

  Use `--scan-interval` to periodically re-scan directories for new SQLite files while the server is running. New databases will be added automatically.

  ```bash
  datasette scan /data --scan-interval 5
  ```

  This re-scans every 5 seconds. Any new SQLite files that appear in the scanned directories will be added to the running Datasette instance without a restart.

  ### Corrupted and locked files

  Since `datasette scan` discovers files automatically, it takes precautions against problematic databases:

  - **Corrupted files are skipped.** Each discovered file is validated by running `SELECT * FROM sqlite_master` before it is served. Files that fail this check are skipped with a warning to stderr.
  - **Locked files are handled safely.** Databases locked by another process are opened in read-only mode instead of causing the server to fail. (The `--nolock` option from `datasette serve` is always enabled and not exposed as a separate flag.)

  These protections apply to files discovered by scanning directories. Explicit file arguments are passed through to `datasette serve` without validation.

  ### Examples

  Scan the current directory and open in a browser:

  ```bash
  datasette scan -o
  ```

  Scan with CORS enabled on a custom port:

  ```bash
  datasette scan /data --cors -p 9000
  ```

  Use `--get` to test without starting a server:

  ```bash
  datasette scan /data --get /.json
  ```

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-scan
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-scan/actions", "Changelog": "https://github.com/simonw/datasette-scan/releases", "Homepage": "https://github.com/simonw/datasette-scan", "Issues": "https://github.com/simonw/datasette-scan/issues"}


name: datasette-schema-versions
summary: Datasette plugin that shows the schema version of every attached database
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-schema-versions
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-schema-versions/
project_url: https://pypi.org/project/datasette-schema-versions/
release_url: https://pypi.org/project/datasette-schema-versions/0.3/
version: 0.3
description:
  # datasette-schema-versions

  [![PyPI](https://img.shields.io/pypi/v/datasette-schema-versions.svg)](https://pypi.org/project/datasette-schema-versions/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-schema-versions?include_prereleases&label=changelog)](https://github.com/simonw/datasette-schema-versions/releases)
  [![Tests](https://github.com/simonw/datasette-schema-versions/workflows/Test/badge.svg)](https://github.com/simonw/datasette-schema-versions/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-schema-versions/blob/main/LICENSE)

  Datasette plugin that shows the schema version of every attached database

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-schema-versions

  ## Usage

  Visit `/-/schema-versions` on your Datasette instance to see a numeric version for the schema for each of your databases.

  Any changes you make to the schema will increase this version number.

project_urls:
  {"CI": "https://github.com/simonw/datasette-schema-versions/actions", "Changelog": "https://github.com/simonw/datasette-schema-versions/releases", "Homepage": "https://github.com/simonw/datasette-schema-versions", "Issues": "https://github.com/simonw/datasette-schema-versions/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-scraper
summary: Adds website scraping abilities to Datasette.
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Colin Dellow
description_content_type: text/markdown
home_page: https://github.com/cldellow/datasette-scraper
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-scraper/
project_url: https://pypi.org/project/datasette-scraper/
release_url: https://pypi.org/project/datasette-scraper/0.5.3/
requires_python: >=3.7
version: 0.5.3
description:
  # datasette-scraper

  [![PyPI](https://img.shields.io/pypi/v/datasette-scraper.svg)](https://pypi.org/project/datasette-scraper/)
  [![Changelog](https://img.shields.io/github/v/release/cldellow/datasette-scraper?include_prereleases&label=changelog)](https://github.com/cldellow/datasette-scraper/releases)
  [![Tests](https://github.com/cldellow/datasette-scraper/workflows/Test/badge.svg)](https://github.com/cldellow/datasette-scraper/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/cldellow/datasette-scraper/blob/main/LICENSE)

  `datasette-scraper` is a Datasette plugin to manage small-ish (~100K pages) crawl and extract jobs.

  - Opinionated yet extensible
    - Some useful tasks are possible out-of-the-box, or write your own pluggy hooks to go further
  - Leans heavily into SQLite
    - Introspect your crawls via ops tables exposed in Datasette
  - Built on robust libraries
    - [Datasette](https://datasette.io/) as a host
    - [selectolax](https://github.com/rushter/selectolax) for HTML parsing
    - [httpx](https://www.python-httpx.org/) for HTTP requests
    - [pluggy](https://pluggy.readthedocs.io/en/stable/) for extensibility
    - [zstandard](https://github.com/indygreg/python-zstandard) for efficiently compressing HTTP responses

  **Not for adversarial crawling**. Want to crawl a site that blocks bots? You're on your own.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-scraper

  ## Usage

  Configure `datasette-scraper` via `metadata.json`. You need to enable the plugin
  on a per-database level.

  To enable it in the `my-database` database, write something like this:

  ```
  {
    "databases": {
      "my-database": {
        "plugins": {
          "datasette-scraper": {
          }
        }
      }
    }
  }
  ```

  The next time you start datasette, the plugin will create several tables in
  the specified database. Go to the `dss_crawl` table to define a crawl.

  A 10-minute end-to-end walkthrough video is available:

  <div align="left">
        <a href="https://www.youtube.com/watch?v=zrSGnz7ErNI">
           <img src="https://img.youtube.com/vi/zrSGnz7ErNI/maxresdefault.jpg" style="width:100%;">
        </a>
  </div>

  ## Usage notes

  `datasette-scraper` requires a database in which to track its operational data,
  and a database in which to store scraped data. They can be the same database.

  Both databases will be put into WAL mode.

  The ops database's `user_version` pragma will be used to track schema versions.

  ## Architecture

  `datasette-scraper` handles the core bookkeeping for scraping--keeping track of
  URLs to be scraped, rate-limiting requests to origins, persisting data into the DB.
  It relies on plugins to do almost all the interesting work. For example, fetching
  the actual pages, following redirects, navigating sitemaps, extracting data.

  The tool comes with plugins for common use cases. Some users may want to author
  their own `after_fetch_url` or `extract_from_response` implementations to do custom
  processing.

  ### Overview

  ```mermaid
  flowchart LR
  direction TB

  subgraph init
    A(user starts crawl) --> B[get_seed_urls]
  end

  subgraph crawl [for each URL to crawl]
    before_fetch_url --> fetch_cached_url --> fetch_url --> after_fetch_url
    fetch_cached_url --> after_fetch_url
  end

  subgraph discover [for each URL crawled]
    discover_urls --> canonicalize_url --> canonicalize_url
    canonicalize_url --> x[queue URL to crawl]
    extract_from_response
  end

  init --> crawl --> discover
  ```

  ### Plugin hooks

  Most plugins will only implement a few of these hooks.

  - `conn` is a read/write `sqlite3.Connection` to the database
  - `config` is the crawl's config

  #### `get_seed_urls(config)`

  Returns a list of strings representing seed URLs to be fetched.

  They will be considered to have depth of 0, i.e. seeds.

  #### `before_fetch_url(conn, config, job_id, url, depth, request_headers)`

  `request_headers` is a dict, you can modify it to control what gets sent in the request.

  Returns:
    - truthy to indicate this URL should not be crawled (for example, crawl max page limit)
    - falsy to express no opinion

  > **Note** `before_fetch_url` vs `canonicalize_url`
  >
  > You can also use the `canonicalize_url` hook to reject URLs prior to them entering
  > the crawl queue.
  >
  > A URL rejected by `canonicalize_url` will not result in an entry in the
  > `dss_crawl_queue` and `dss_crawl_queue_history` tables.
  >
  > Which one you use is a matter of taste, in general, if you _never_ want the URL,
  > reject it at canonicalization time.

  #### `fetch_cached_url(conn, config, url, depth, request_headers)`

  Fetch a previously-cached HTTP response. The system will not have checked that
  there was rate limit available before calling this.

  Returns:
    - `None`, to indicate not handled
    - a response object, which is a dict with:
      - `fetched_at` - an ISO 8601 time like `2022-12-26 01:23:45.00`
      - `headers` - the response headers, eg `[['content-type', 'text/html']]`
      - `status_code` - the respones code, eg `200`
      - `text` - the response body

  Once any plugin has returned a truthy value, no other plugin's `fetch_url`
  hook will be invoked.


  #### `fetch_url(conn, config, url, request_headers)`

  Fetch an HTTP response from the live server. The system will have checked that there
  was rate limit available before calling this.

  Same return type and behaviour as `fetch_cached_url`.

  #### `after_fetch_url(conn, config, url, request_headers, response, fresh, fetch_duration)`

  Do something with a fetched URL.

  #### `discover_urls(config, url, response)`

  Returns a list of URLs to crawl.

  The URLs can be either strings, in which case they'll get enqueued as depth + 1, or tuple of URL and depth. This can be useful for paginated index pages, where you'd like to crawl to a max depth of, say, 2, but treat all the index pages as being at depth 1.

  #### `canonicalize_url(config, from_url, to_url, to_url_depth)`

  Returns:
    - `False` to filter URL
    - an URL to be crawled instead
    - `None` or `True` to no-op

  The URL to be crawled can be a string, or a tuple of string and depth.

  This hook is useful for:
    - blocking URLs that we never want
    - canonicalizing URLs, for example, by omitting query parameters
    - restricting crawls to same origin
    - resetting depth for pagination

  #### `extract_from_response(config, url, response)`

  Returns an object of rows-to-be-inserted-or-upserted:

  ```jsonc
  {
    "dbname": {  // can be omitted, in which case, current DB will be used
      "users": [
        {
          "id!": "cldellow@gmail.com",  // ! indicates pkey, compound OK
          "name": "Colin",
        },
        {
          "id!": "santa@northpole.com",
          "name": "Santa Claus",
        }
      ],
      "places": [
        {
          "id@": "santa@northpole.com",
          "__delete": true
        },
        {
          "id@": "cldellow@gmail.com",
          "city": "Kitchener",
        },
        {
          "id@": "cldellow@gmail.com",
          "city": "Dawson Creek"
        }
      ]
    }
  }
  ```

  Column names can have sigils at the end:
  - `!` says the column is part of the pkey; there can be at most 1 row with this value
  - `@` says the column should be indexed; there can be multiple rows with this value

  Columns with sigils must be known at table creation time. Although you can have
  multiple columns with sigils, you cannot mix `!` and `@` sigils in the same table.

  Any missing tables or columns will be created. Columns will have `ANY` data type.
  Columns will be nullable unless they have the `!` sigil.

  You can indicate that a row should be deleted by emitting `__delete` key in your object.

  `datasette-scraper` may commit your changes to the database in batches in order to
  reduce write transactions and improve throughput. It may also elide
  DELETE/INSERT statements entirely if it determines that the state of the database
  would be unchanged.

  If you'd like to control the schema more carefully, please create the table manually.

  #### Metadata hooks

  These hooks don't affect operation of the scrapes. They provide metadata to
  help validate a user's configuration and show UI to configure a crawl.

  ##### config_schema()

  Returns a `ConfigSchema` option that defines how this plugin is configured.

  Configuration is done via [JSON schema](https://json-schema.org/understanding-json-schema/). UI is done via [JSON Forms](https://jsonforms.io/).

  Look at the existing plugins to learn how to use this hook.

  The schema is optional; if omitted, you will need to configure the plug in
  out of band.

  ##### config_default_value()

  Returns `None` to indicate that new crawls should not use this plugin by default.

  Otherwise, returns a reasonable default value that conforms to the schema in `config_schema()`

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-scraper
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/cldellow/datasette-scraper/actions", "Changelog": "https://github.com/cldellow/datasette-scraper/releases", "Homepage": "https://github.com/cldellow/datasette-scraper", "Issues": "https://github.com/cldellow/datasette-scraper/issues"}

requires_dist:
  ["datasette", "selectolax", "datasette-template-sql", "pluggy", "httpx", "zstandard", "more-itertools", "wheel ; extra == 'test'", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-watch ; extra == 'test'", "coverage ; extra == 'test'"]


name: datasette-seaborn
summary: Statistical visualizations for Datasette using Seaborn
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-seaborn
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-seaborn/
project_url: https://pypi.org/project/datasette-seaborn/
release_url: https://pypi.org/project/datasette-seaborn/0.2a0/
version: 0.2a0
description:
  # datasette-seaborn

  [![PyPI](https://img.shields.io/pypi/v/datasette-seaborn.svg)](https://pypi.org/project/datasette-seaborn/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-seaborn?include_prereleases&label=changelog)](https://github.com/simonw/datasette-seaborn/releases)
  [![Tests](https://github.com/simonw/datasette-seaborn/workflows/Test/badge.svg)](https://github.com/simonw/datasette-seaborn/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-seaborn/blob/main/LICENSE)

  Statistical visualizations for Datasette using Seaborn

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-seaborn

  ## Usage

  Navigate to the new `.seaborn` extension for any Datasette table.

  The `_seaborn` argument specifies a method on `sns` to execute, e.g. `?_seaborn=relplot`.

  Extra arguments to those methods can be specified using e.g. `&_seaborn_x=column_name`.

  ## Configuration

  The plugin implements a default rendering time limit of five seconds. You can customize this limit using the `render_time_limit` setting, which accepts a floating point number of seconds. Add this to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-seaborn": {
              "render_time_limit": 1.0
          }
      }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-seaborn
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-seaborn/actions", "Changelog": "https://github.com/simonw/datasette-seaborn/releases", "Homepage": "https://github.com/simonw/datasette-seaborn", "Issues": "https://github.com/simonw/datasette-seaborn/issues"}

requires_dist:
  ["datasette (>=0.50)", "seaborn (>=0.11.0)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-search-all
summary: Datasette plugin for searching all searchable tables at once
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-search-all
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-search-all/
project_url: https://pypi.org/project/datasette-search-all/
provides_extra: ["playwright", "test"]
release_url: https://pypi.org/project/datasette-search-all/1.1.4/
requires_python: >=3.7
version: 1.1.4
description:
  # datasette-search-all

  [![PyPI](https://img.shields.io/pypi/v/datasette-search-all.svg)](https://pypi.org/project/datasette-search-all/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-search-all?include_prereleases&label=changelog)](https://github.com/simonw/datasette-search-all/releases)
  [![Tests](https://github.com/simonw/datasette-search-all/workflows/Test/badge.svg)](https://github.com/simonw/datasette-search-all/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-search-all/blob/main/LICENSE)

  Datasette plugin for searching all searchable tables at once.

  ## Installation

  Install the plugin in the same Python environment as Datasette:

      pip install datasette-search-all

  ## Background

  See [datasette-search-all: a new plugin for searching multiple Datasette tables at once](https://simonwillison.net/2020/Mar/9/datasette-search-all/) for background on this project. You can try the plugin out at https://fara.datasettes.com/

  ## Usage

  This plugin only works if at least one of the tables connected to your Datasette instance has been configured for SQLite's full-text search.

  The [Datasette search documentation](https://docs.datasette.io/en/stable/full_text_search.html) includes details on how to enable full-text search for a table.

  You can also use the following tools:

  * [sqlite-utils](https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search) includes a command-line tool for enabling full-text search.
  * [datasette-enable-fts](https://github.com/simonw/datasette-enable-fts) is a Datasette plugin that adds a web interface for enabling search for specific columns.

  If the plugin detects at least one searchable table it will add a search form to the homepage.

  You can also navigate to `/-/search` on your Datasette instance to use the search interface directly.

  ## Screenshot

  ![Animated screenshot showing the plugin in action](https://raw.githubusercontent.com/simonw/datasette-search-all/main/animated-screenshot.gif)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-search-all
  python -m venv venv
  source venv/bin/activate
  ```
  Or if you are using `pipenv`:
  ```bash
  pipenv shell
  ```
  Now install the dependencies and tests:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```
  To run the browser automation tests:
  ```bash
  pip install -e '.[test,playwright]'
  pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-search-all/actions", "Changelog": "https://github.com/simonw/datasette-search-all/releases", "Homepage": "https://github.com/simonw/datasette-search-all", "Issues": "https://github.com/simonw/datasette-search-all/issues"}

requires_dist:
  ["datasette>=0.63.1", "pytest-playwright; extra == \"playwright\"", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "sqlite-utils; extra == \"test\"", "nest-asyncio; extra == \"test\""]


name: datasette-secret-santa
summary: Run a secret santa using Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-secret-santa
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-secret-santa/
project_url: https://pypi.org/project/datasette-secret-santa/
release_url: https://pypi.org/project/datasette-secret-santa/0.1/
requires_dist: ["datasette", "cryptography", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-secret-santa

  [![PyPI](https://img.shields.io/pypi/v/datasette-secret-santa.svg)](https://pypi.org/project/datasette-secret-santa/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-secret-santa?include_prereleases&label=changelog)](https://github.com/simonw/datasette-secret-santa/releases)
  [![Tests](https://github.com/simonw/datasette-secret-santa/workflows/Test/badge.svg)](https://github.com/simonw/datasette-secret-santa/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-secret-santa/blob/main/LICENSE)

  Run a secret santa using Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-secret-santa

  ## Running this on Glitch

  https://glitch.com/ is a great way to host this application.

  You can [remix this project on Glitch](https://glitch.com/~datasette-secret-santa) to get your own copy.

  Or you can create a new application and paste the following into your `glitch.json` file:

  ```json
  {
    "install": "pip3 install --user datasette datasette-secret-santa -U",
    "start": "datasette --create .data/santa.db -p 3000"
  }
  ```
  Then visit your new app's homepage and click the link to create a new secret santa group.

  ## Usage

  ![Animated GIF showing the plugin in action - the user adds three names, then gets the password for their account - then hits the assign button and uses their password to find out who they have been assigned.](https://raw.githubusercontent.com/simonw/datasette-secret-santa/main/secret-santa.gif)

  This plugin requires a database called `santa.db`. You can run it and create such a database like this:

      datasette santa.db --create

  It expects to be the only plugin installed, and will take over the `/` homepage.

  To create a new Secret Santa, visit `/santa/create_secret_santa` (linked from the homepage).

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-secret-santa
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-secret-santa/actions", "Changelog": "https://github.com/simonw/datasette-secret-santa/releases", "Homepage": "https://github.com/simonw/datasette-secret-santa", "Issues": "https://github.com/simonw/datasette-secret-santa/issues"}


name: datasette-secrets
summary: Manage secrets such as API keys for use with other Datasette plugins
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Datasette
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-secrets/
project_url: https://pypi.org/project/datasette-secrets/
release_url: https://pypi.org/project/datasette-secrets/0.2/
requires_python: >=3.8
version: 0.2
description:
  # datasette-secrets

  [![PyPI](https://img.shields.io/pypi/v/datasette-secrets.svg)](https://pypi.org/project/datasette-secrets/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-secrets?include_prereleases&label=changelog)](https://github.com/datasette/datasette-secrets/releases)
  [![Tests](https://github.com/datasette/datasette-secrets/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-secrets/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-secrets/blob/main/LICENSE)

  Manage secrets such as API keys for use with other Datasette plugins

  This plugin requires a **Datasette 1.0 alpha** release.

  Datasette plugins sometimes need access to secrets, such as API keys used to integrate with tools hosted outside of Datasette - things like geocoders or hosted AI language models.

  This plugin provides ways to configure those secrets:

  - Secrets can be configured using environment variables, such as `DATASETTE_SECRETS_OPENAI_API_KEY`
  - Secrets can be stored, encrypted, in a SQLite database table which administrator users can then update through the Datasette web interface

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-secrets
  ```
  ## Configuration

  First you will need to generate an encryption key for this plugin to use. Run this command:

  ```bash
  datasette secrets generate-encryption-key
  ```
  Store this secret somewhere secure. It will be used to both encrypt and decrypt secrets stored by this plugin - if you lose it you will not be able to recover your secrets.

  Configure the plugin with these these two plugin settings:

  ```yaml
  plugins:
    datasette-secrets:
      encryption_key:
        $env: DATASETTE_SECRETS_ENCRYPTION_KEY
      database: name_of_database
  ```
  The `encryption_key` setting should be set to the encryption key you generated earlier. You can store it in an environment variable if you prefer.

  `database` is the name of the database that the encrypted keys should be stored in. Omit this setting to use the internal database.

  ### Using the internal database

  While the secrets stored in the `datasette_secrets` table are encrypted, we still recommend hiding that table from view.

  One way to do that is to keep the table in Datasette's internal database, which is invisible to all users, even users who are logged in.

  By default, the internal database is an in-memory database that is reset when Datasette restarts. This is no good for persistent secret storage!

  Instead, you should switch Datasette to using an on-disk internal database. You can do this by starting Datasette with the `--internal` option:
  ```bash
  datasette data.db --internal internal.db
  ```
  Your secrets will be stored in the `datasette_secrets` table in that database file.

  ### Permissions

  Only users with the `manage-secrets` permission will have access to manage secrets through the Datasette web interface.

  You can grant that permission to the `root` user (or the user with an ID of your choice) by including this in your `datasette.yml` file:

  ```yaml
  permissions:
    manage-secrets:
      id: root
  ```
  Then start Datasette like this (with `--root` to get a URL to login as the root user):
  ```bash
  datasette data.db --internal internal.db -c datasette.yml --root
  ```
  Alternatively, use the `-s` option to set that setting without creating a configuration file:
  ```bash
  datasette data.db --internal internal.db \
    -s permissions.manage-secrets.id root \
    --root
  ```

  ## Usage

  users with the `manage-secrets` permission will see a new "Manage secrets" link in the Datasette navigation menu. This interface can also be accessed at `/-/secrets`.

  The page with the list of secrets will show the user who last updated each secret. This will use the [actors_from_ids()](https://docs.datasette.io/en/latest/plugin_hooks.html#actors-from-ids-datasette-actor-ids) mechanism, displaying the actor's `username` if available, otherwise the `name`, otherwise the `id`.

  ## For plugin authors

  Plugins can depend on this plugin if they want to implement secrets.

  `datasette-secrets` to the `dependencies` list in `pyproject.toml`.

  Then declare the name and description of any secrets you need using the `register_secrets()` plugin hook:

  ```python
  from datasette import hookimpl
  from datasette_secrets import Secret

  @hookimpl
  def register_secrets():
      return [
          Secret(
              name="OPENAI_API_KEY",
              description="An OpenAI API key"
          ),
      ]
  ```
  You can also provide optional `obtain_url` and `obtain_label` fields to link to a page where a user can obtain an API key:
  ```python
  @hookimpl
  def register_secrets():
      return [
          Secret(
              name="OPENAI_API_KEY",
              obtain_url="https://platform.openai.com/api-keys",
              obtain_label="Get an OpenAI API key"
          ),
      ]
  ```

  The hook can take an optional `datasette` argument. It can return a list or an `async def` function that, when awaited, returns a list.

  The list should consist of `Secret()` instances, each with a name and an optional description. The description can contain HTML.

  To obtain the current value of the secret, use the `await get_secret()` method:

  ```python
  from datasette_secrets import get_secret

  # Third argument is the actor_id, optional
  secret = await get_secret(datasette, "OPENAI_API_KEY", "root")
  ```
  If the Datasette administrator set a `DATASETTE_SECRETS_OPENAI_API_KEY` environment variable, that will be returned.

  Otherwise the encrypted value in the database table will be decrypted and returned - or `None` if there is no configured secret.

  The `last_used_at` column is updated every time a secret is accessed. The `last_used_by` column will be set to the actor ID passed to `get_secret()`, or `null` if no actor ID was passed.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-secrets
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-secrets/actions", "Changelog": "https://github.com/datasette/datasette-secrets/releases", "Homepage": "https://github.com/datasette/datasette-secrets", "Issues": "https://github.com/datasette/datasette-secrets/issues"}

requires_dist:
  ["datasette", "cryptography", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "datasette-test>=0.3.2; extra == \"test\""]


name: datasette-sentry
summary: Datasette plugin for configuring Sentry
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-sentry
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sentry/
project_url: https://pypi.org/project/datasette-sentry/
project_urls: {"Homepage": "https://github.com/simonw/datasette-sentry"}
release_url: https://pypi.org/project/datasette-sentry/0.4/
requires_dist: ["sentry-sdk", "datasette >=0.62", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
version: 0.4
classifiers:
  ["License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8"]

description:
  # datasette-sentry

  [![PyPI](https://img.shields.io/pypi/v/datasette-sentry.svg)](https://pypi.org/project/datasette-sentry/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-sentry?include_prereleases&label=changelog)](https://github.com/simonw/datasette-sentry/releases)
  [![Tests](https://github.com/simonw/datasette-sentry/workflows/Test/badge.svg)](https://github.com/simonw/datasette-sentry/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-sentry/blob/main/LICENSE)

  Datasette plugin for configuring Sentry for error reporting
   
  ## Installation
  ```bash
  pip install datasette-sentry
  ```
  ## Usage

  This plugin only takes effect if your `metadata.json` file contains relevant top-level plugin configuration in a `"datasette-sentry"` configuration key.

  You will need a Sentry DSN - see their [Getting Started instructions](https://docs.sentry.io/error-reporting/quickstart/?platform=python).

  Add it to `metadata.json` like this:

  ```json
  {
      "plugins": {
          "datasette-sentry": {
              "dsn": "https://KEY@sentry.io/PROJECTID"
          }
      }
  }
  ```
  Settings in `metadata.json` are visible to anyone who visits the `/-/metadata` URL so this is a good place to take advantage of Datasette's [secret configuration values](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values), in which case your configuration will look more like this:
  ```json
  {
      "plugins": {
          "datasette-sentry": {
              "dsn": {
                  "$env": "SENTRY_DSN"
              }
          }
      }
  }
  ```
  Then make a `SENTRY_DSN` environment variable available to Datasette.

  ## Configuration

  In addition to the `dsn` setting, you can also configure the Sentry [sample rate](https://docs.sentry.io/platforms/python/configuration/sampling/) by setting  `sample_rate` to a floating point number between 0 and 1.

  For example, to capture 25% of errors you would do this:

  ```json
  {
      "plugins": {
          "datasette-sentry": {
              "dsn": {
                  "$env": "SENTRY_DSN"
              },
              "sample_rate": 0.25
          }
      }
  }
  ```

  ### Performance monitoring

  Sentry [Performance Monitoring](https://docs.sentry.io/product/performance/) records full traces of page for further analysis, in addition to tracking errors.

  You can enable that by adding "enable_tracing" to your plugin configuration:

  ```json
  {
      "plugins": {
          "datasette-sentry": {
              "dsn": {
                  "$env": "SENTRY_DSN"
              },
              "enable_tracing": true
          }
      }
  }
  ```
  The default sample rate if you do this will be `1.0`, meaning every response will be traced. This can get expensive - you can adjust the tracing rate using `traces_sample_rate`. Set that to `0.1` to sample 10% of requests, for example:

  ```json
  {
      "plugins": {
          "datasette-sentry": {
              "dsn": {
                  "$env": "SENTRY_DSN"
              },
              "enable_tracing": true,
              "traces_sample_rate": 0.1
          }
      }
  }
  ```


name: datasette-show-errors
summary: Datasette plugin for displaying error tracebacks
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-show-errors
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-show-errors/
project_url: https://pypi.org/project/datasette-show-errors/
project_urls: {"Homepage": "https://github.com/simonw/datasette-show-errors"}
release_url: https://pypi.org/project/datasette-show-errors/0.2/
requires_dist: ["starlette", "datasette", "pytest ; extra == 'test'"]
version: 0.2
description:
  # datasette-show-errors

  [![PyPI](https://img.shields.io/pypi/v/datasette-show-errors.svg)](https://pypi.org/project/datasette-show-errors/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-show-errors.svg?style=svg)](https://circleci.com/gh/simonw/datasette-show-errors)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-show-errors/blob/master/LICENSE)

  Datasette plugin for displaying error tracebacks.

  ## Installation

      pip install datasette-show-errors

  ## Usage

  Installing the plugin will cause any internal error to be displayed with a full traceback, rather than just a generic 500 page.

  Be careful not to use this in a context that might expose sensitive information.


name: datasette-showboat
summary: Datasette plugin for SHOWBOAT_REMOTE_URL
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-showboat/
project_url: https://pypi.org/project/datasette-showboat/
release_url: https://pypi.org/project/datasette-showboat/0.1a2/
requires_dist: ["datasette>=1.0a24"]
requires_python: >=3.10
version: 0.1a2
description:
  # datasette-showboat

  [![PyPI](https://img.shields.io/pypi/v/datasette-showboat.svg)](https://pypi.org/project/datasette-showboat/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-showboat?include_prereleases&label=changelog)](https://github.com/simonw/datasette-showboat/releases)
  [![Tests](https://github.com/simonw/datasette-showboat/actions/workflows/test.yml/badge.svg)](https://github.com/simonw/datasette-showboat/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-showboat/blob/main/LICENSE)

  Datasette plugin that provides a remote viewer for [Showboat](https://github.com/simonw/showboat) documents. It receives streaming document chunks over HTTP and displays them in a live-updating web interface.

  See [this blog post](https://simonwillison.net/2026/Feb/17/chartroom-and-datasette-showboat/#datasette-showboat) for background on this project.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-showboat
  ```

  ## Usage

  Once installed, the plugin adds a `/-/showboat` page to your Datasette instance listing all received documents, and a `/-/showboat/receive` endpoint for ingesting chunks.

  ### Sending documents

  Set the `SHOWBOAT_REMOTE_URL` environment variable to point at your Datasette instance:

  ```bash
  export SHOWBOAT_REMOTE_URL="https://your-datasette-instance/-/showboat/receive"
  ```

  The `/-/showboat` page will display the correct URL for your instance including the hostname.

  ### Permissions

  Viewing showboat documents requires the `showboat` permission. By default this is **denied** to anonymous users — only the root user (when Datasette is started with `--root`) has access automatically.

  To grant access to specific users, add to your `datasette.yaml`:

  ```yaml
  permissions:
    showboat:
      id: your-username
  ```

  Or to allow all authenticated users:

  ```yaml
  permissions:
    showboat:
      id: "*"
  ```

  The receive endpoint (`/-/showboat/receive`) does not require the `showboat` permission — it uses token authentication instead (see below).

  ### Token authentication

  To protect the receive endpoint, configure a secret token in your `datasette.yaml` (or `metadata.yaml`):

  ```yaml
  plugins:
    datasette-showboat:
      token: your-secret-token
  ```

  When a token is configured, all requests to `/-/showboat/receive` must include it as a query parameter:

  ```bash
  export SHOWBOAT_REMOTE_URL="https://your-datasette-instance/-/showboat/receive?token=your-secret-token"
  ```

  Without a configured token, the receive endpoint accepts all POST requests.

  ### Custom database

  By default chunks are stored in Datasette's internal database. To use a named database instead:

  ```yaml
  plugins:
    datasette-showboat:
      database: my_database
  ```

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-showboat
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-showboat/actions", "Changelog": "https://github.com/simonw/datasette-showboat/releases", "Homepage": "https://github.com/simonw/datasette-showboat", "Issues": "https://github.com/simonw/datasette-showboat/issues"}


name: datasette-simple-html
summary: Datasette SQL functions for very simple HTML operations
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-simple-html
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-simple-html/
project_url: https://pypi.org/project/datasette-simple-html/
release_url: https://pypi.org/project/datasette-simple-html/0.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.2
description:
  # datasette-simple-html

  [![PyPI](https://img.shields.io/pypi/v/datasette-simple-html.svg)](https://pypi.org/project/datasette-simple-html/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-simple-html?include_prereleases&label=changelog)](https://github.com/simonw/datasette-simple-html/releases)
  [![Tests](https://github.com/simonw/datasette-simple-html/workflows/Test/badge.svg)](https://github.com/simonw/datasette-simple-html/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-simple-html/blob/main/LICENSE)

  Datasette SQL functions for very simple HTML operations

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-simple-html

  ## Usage

  This plugin provides the following SQL functions:

  ### html_strip_tags(text)

  Returns the text with any `<...>` tags removed.

  ### html_escape(text)

  Escapes any HTML special characters in the text, e.g. `>` becomes `&gt;`. Uses [html.escape(text, quote=True)](https://docs.python.org/3/library/html.html#html.escape) from the Python standard library.

  ### html_unescape(text)

  Unescapes any HTML special characters, so `&gt;` becomes `>`. Also handles numeric entities, so `&#x27;` becomes `'`. Uses [html.unescape(text)](https://docs.python.org/3/library/html.html#html.unescape) from the Python standard library.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-simple-html
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-simple-html/actions", "Changelog": "https://github.com/simonw/datasette-simple-html/releases", "Homepage": "https://github.com/simonw/datasette-simple-html", "Issues": "https://github.com/simonw/datasette-simple-html/issues"}


name: datasette-sitemap
summary: Generate sitemap.xml for Datasette sites
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-sitemap
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sitemap/
project_url: https://pypi.org/project/datasette-sitemap/
release_url: https://pypi.org/project/datasette-sitemap/1.0/
requires_python: >=3.7
version: 1.0
description:
  # datasette-sitemap

  [![PyPI](https://img.shields.io/pypi/v/datasette-sitemap.svg)](https://pypi.org/project/datasette-sitemap/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-sitemap?include_prereleases&label=changelog)](https://github.com/simonw/datasette-sitemap/releases)
  [![Tests](https://github.com/simonw/datasette-sitemap/workflows/Test/badge.svg)](https://github.com/simonw/datasette-sitemap/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-sitemap/blob/main/LICENSE)

  Generate sitemap.xml for Datasette sites

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-sitemap

  ## Demo

  This plugin is used for the sitemap on [til.simonwillison.net](https://til.simonwillison.net/):

  - https://til.simonwillison.net/sitemap.xml

  Here's [the configuration](https://github.com/simonw/til/blob/d4f67743a90a67100b46145986b2dec6f8d96583/metadata.yaml#L14-L16) used for that sitemap.

  ## Usage

  Once configured, this plugin adds a sitemap at `/sitemap.xml` with a list of URLs.

  This list is defined using a SQL query in `metadata.json` (or `.yml`) that looks like this:

  ```json
  {
    "plugins": {
      "datasette-sitemap": {
        "query": "select '/' || id as path from my_table"
      }
    }
  }
  ```

  Using `metadata.yml` allows for multi-line SQL queries which can be easier to maintain:

  ```yaml
  plugins:
    datasette-sitemap:
      query: |
        select
          '/' || id as path
        from
          my_table
  ```
  The SQL query must return a column called `path`. The values in this column must begin with a `/`. They will be used to generate a sitemap that looks like this:

  ```xml
  <?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url><loc>https://example.com/1</loc></url>
    <url><loc>https://example.com/2</loc></url>
  </urlset>
  ```
  You can use ``UNION`` in your SQL query to combine results from multiple tables, or include literal paths that you want to include in the index:

  ```sql
  select
    '/data/table1/' || id as path
    from table1
  union
  select
    '/data/table2/' || id as path
    from table2
  union
  select
    '/about' as path
  ```
  If your Datasette instance has multiple databases you can configure the database to query using the `database` configuration property.

  By default the domain name for the genearted URLs in the sitemap will be detected from the incoming request.

  You can set `base_url` instead to override this. This should not include a trailing slash.

  This example shows both of those settings, running the query against the `content` database and setting a custom base URL:

  ```yaml
  plugins:
    datasette-sitemap:
      query: |
        select '/plugins/' || name as path from plugins
        union
        select '/tools/' || name as path from tools
        union
        select '/news' as path
      database: content
      base_url: https://datasette.io
  ```
  [Try that query](https://datasette.io/content?sql=select+%27%2Fplugins%2F%27+||+name+as+path+from+plugins%0D%0Aunion%0D%0Aselect+%27%2Ftools%2F%27+||+name+as+path+from+tools%0D%0Aunion%0D%0Aselect+%27%2Fnews%27+as+path%0D%0A).

  ## robots.txt

  This plugin adds a `robots.txt` file pointing to the sitemap:

  ```
  Sitemap: http://example.com/sitemap.xml
  ```

  You can take full control of the sitemap by installing and configuring the [datasette-block-robots](https://datasette.io/plugins/datasette-block-robots) plugin.

  This plugin will add the `Sitemap:` line even if you are using `datasette-block-robots` for the rest of your `robots.txt` file.

  ## Adding paths to the sitemap from other plugins

  This plugin adds a new [plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html) to Datasete called `sitemap_extra_paths()` which can be used by other plugins to add their own additional lines to the `sitemap.xml` file.

  The hook accepts these optional parameters:

  - `datasette`: The current [Datasette instance](https://docs.datasette.io/en/stable/internals.html#datasette-class). You can use this to execute SQL queries or read plugin configuration settings.
  - `request`: The [Request object](https://docs.datasette.io/en/stable/internals.html#request-object) representing the incoming request to `/sitemap.xml`.

  The hook should return a list of strings, each representing a path to be added to the sitemap. Each path must begin with a `/`.

  It can also return an `async def` function, which will be awaited and used to generate a list of lines. Use this option if you need to make `await` calls inside you hook implementation.

  This example uses the hook to add two extra paths, one of which came from a SQL query:

  ```python
  from datasette import hookimpl

  @hookimpl
  def sitemap_extra_paths(datasette):
      async def inner():
          db = datasette.get_database()
          path_from_db = (await db.execute("select '/example'")).single_value()
          return ["/about", path_from_db]
      return inner
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-sitemap
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-sitemap/actions", "Changelog": "https://github.com/simonw/datasette-sitemap/releases", "Homepage": "https://github.com/simonw/datasette-sitemap", "Issues": "https://github.com/simonw/datasette-sitemap/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "datasette-block-robots ; extra == 'test'"]


name: datasette-socrata
summary: Import data from Socrata into Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-socrata
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-socrata/
project_url: https://pypi.org/project/datasette-socrata/
release_url: https://pypi.org/project/datasette-socrata/0.3.1/
requires_python: >=3.7
version: 0.3.1
description:
  # datasette-socrata

  [![PyPI](https://img.shields.io/pypi/v/datasette-socrata.svg)](https://pypi.org/project/datasette-socrata/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-socrata?include_prereleases&label=changelog)](https://github.com/simonw/datasette-socrata/releases)
  [![Tests](https://github.com/simonw/datasette-socrata/workflows/Test/badge.svg)](https://github.com/simonw/datasette-socrata/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-socrata/blob/main/LICENSE)

  Import data from Socrata into Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-socrata

  ## Usage

  Make sure you have [enabled WAL mode](https://til.simonwillison.net/sqlite/enabling-wal-mode) on your database files before using this plugin.

  Once installed, an interface for importing data from Socrata will become available at this URL:

      /-/import-socrata

  Users will be able to paste in a URL to a dataset on Socrata in order to initialize an import.

  You can also pre-fill the form by passing a `?url=` parameter, for example:

      /-/import-socrata?url=https://data.sfgov.org/City-Infrastructure/Street-Tree-List/tkzw-k3nq

  Any database that is attached to Datasette, is NOT loaded as immutable (with the `-i` option) and that has WAL mode enabled will be available for users to import data into.

  The `import-socrata` permission governs access. By default the `root` actor (accessible using `datasette --root` to start Datasette) is granted that permission.

  You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to other users.

  ## Configuration

  If you only want Socrata imports to be allowed to a specific database, you can configure that using plugin configration in `metadata.yml`:

  ```yaml
  plugins:
    datasette-socrata:
      database: socrata
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-socrata
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-socrata/actions", "Changelog": "https://github.com/simonw/datasette-socrata/releases", "Homepage": "https://github.com/simonw/datasette-socrata", "Issues": "https://github.com/simonw/datasette-socrata/issues"}

requires_dist:
  ["datasette", "sqlite-utils (>=3.27)", "datasette-low-disk-space-hook", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-httpx ; extra == 'test'"]


name: datasette-sqlite-authorizer
summary: Configure Datasette to block operations using the SQLite set_authorizer mechanism
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-sqlite-authorizer/
project_url: https://pypi.org/project/datasette-sqlite-authorizer/
release_url: https://pypi.org/project/datasette-sqlite-authorizer/0.2/
requires_python: >=3.8
version: 0.2
description:
  # datasette-sqlite-authorizer

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-authorizer.svg)](https://pypi.org/project/datasette-sqlite-authorizer/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-sqlite-authorizer?include_prereleases&label=changelog)](https://github.com/datasette/datasette-sqlite-authorizer/releases)
  [![Tests](https://github.com/datasette/datasette-sqlite-authorizer/workflows/Test/badge.svg)](https://github.com/datasette/datasette-sqlite-authorizer/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-sqlite-authorizer/blob/main/LICENSE)

  Configure Datasette to block operations using the SQLite set_authorizer mechanism

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-sqlite-authorizer
  ```
  ## Usage

  This plugin currently offers a single configuration option: `read_only_tables`. You can use this to specify a list of tables that should be read-only.

  If a table is read-only, any attempt to write to it - `insert`, `update`, `delete`, `drop table`, `alter table` - will be denied with an error message.

  To configure read-only tables, add the following to your `metadata.yaml` file:

  ```yaml
  plugins:
    datasette-sqlite-authorizer:
      read_only_tables:
      - table: my_table
        database: my_database
  ```
  You can omit the `database` key if you want to apply the same rule to all databases.

  Here's how to use this plugin to make all tables relating to [Litestream](https://litestream.io/) synchronization read-only, across all attached databases:

  ```yaml
  plugins:
    datasette-sqlite-authorizer:
      read_only_tables:
      - table: _litestream_lock
      - table: _litestream_seq
  ```

  ## Logging actions

  To enable debug output (to standard error) for specific action checks, add them to a `debug_actions` list. For example, to log all `SQLITE_INSERT` actions:

  ```yaml
  plugins:
    datasette-sqlite-authorizer:
      debug_actions:
      - SQLITE_INSERT
      - SQLITE_DELETE
  ```
  Logged output will look something like this:
  ```
  authorizer: {"action": "SQLITE_INSERT", "arg1": "foo", "arg2": null, "db_name": "main", "trigger_name": null, "result": "OK"}
  authorizer: {"action": "SQLITE_DELETE", "arg1": "foo", "arg2": null, "db_name": "main", "trigger_name": null, "result": "OK"}
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-sqlite-authorizer
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-sqlite-authorizer/actions", "Changelog": "https://github.com/simonw/datasette-sqlite-authorizer/releases", "Homepage": "https://github.com/simonw/datasette-sqlite-authorizer", "Issues": "https://github.com/simonw/datasette-sqlite-authorizer/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "datasette-test ; extra == 'test'"]


name: datasette-sqlite-debug-authorizer
summary: Debug SQLite authorizer calls
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-sqlite-debug-authorizer/
project_url: https://pypi.org/project/datasette-sqlite-debug-authorizer/
release_url: https://pypi.org/project/datasette-sqlite-debug-authorizer/0.1.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.8
version: 0.1.1
description:
  # datasette-sqlite-debug-authorizer

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-debug-authorizer.svg)](https://pypi.org/project/datasette-sqlite-debug-authorizer/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-sqlite-debug-authorizer?include_prereleases&label=changelog)](https://github.com/datasette/datasette-sqlite-debug-authorizer/releases)
  [![Tests](https://github.com/datasette/datasette-sqlite-debug-authorizer/workflows/Test/badge.svg)](https://github.com/datasette/datasette-sqlite-debug-authorizer/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-sqlite-debug-authorizer/blob/main/LICENSE)

  Debug SQLite authorizer calls

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-sqlite-debug-authorizer

  ## Usage

  Once installed, every SQLite permission check using the `conn.set_authorizer()` API will be logged to standard error.

  This is useful for if you want to use that authorizer API to implement your own custom permissions, and need to see what calls are being made to the authorizer.

  Example output (it's generally a lot noisier than this) for `select * from sqlite_master`:

  ```
  SQLITE_SELECT: 
  SQLITE_READ:  table="sqlite_master" column="type" db_name=main
  SQLITE_READ:  table="sqlite_master" column="name" db_name=main
  SQLITE_READ:  table="sqlite_master" column="tbl_name" db_name=main
  SQLITE_READ:  table="sqlite_master" column="rootpage" db_name=main
  SQLITE_READ:  table="sqlite_master" column="sql" db_name=main
  ```
  See [sqlite-authorizer-examples](https://github.com/simonw/sqlite-authorizer-examples) for more detailed documentation on what to expect from these calls.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-sqlite-debug-authorizer
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-sqlite-debug-authorizer/actions", "Changelog": "https://github.com/simonw/datasette-sqlite-debug-authorizer/releases", "Homepage": "https://github.com/simonw/datasette-sqlite-debug-authorizer", "Issues": "https://github.com/simonw/datasette-sqlite-debug-authorizer/issues"}


name: datasette-sqlite-fastrand
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-fastrand
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-fastrand/
project_url: https://pypi.org/project/datasette-sqlite-fastrand/
release_url: https://pypi.org/project/datasette-sqlite-fastrand/0.2.1/
requires_dist: ["datasette", "sqlite-fastrand", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.2.1
project_urls:
  {"CI": "https://github.com/asg017/sqlite-fastrand/actions", "Changelog": "https://github.com/asg017/sqlite-fastrand/releases", "Homepage": "https://github.com/asg017/sqlite-fastrand", "Issues": "https://github.com/asg017/sqlite-fastrand/issues"}


name: datasette-sqlite-fts4
summary: Datasette plugin exposing SQL functions from sqlite-fts4
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-sqlite-fts4
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-fts4/
project_url: https://pypi.org/project/datasette-sqlite-fts4/
release_url: https://pypi.org/project/datasette-sqlite-fts4/0.3.2/
requires_python: >=3.7
version: 0.3.2
description:
  # datasette-sqlite-fts4

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-fts4.svg)](https://pypi.org/project/datasette-sqlite-fts4/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-sqlite-fts4?include_prereleases&label=changelog)](https://github.com/simonw/datasette-sqlite-fts4/releases)
  [![Tests](https://github.com/simonw/datasette-sqlite-fts4/workflows/Test/badge.svg)](https://github.com/simonw/datasette-sqlite-fts4/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-sqlite-fts4/blob/main/LICENSE)

  Datasette plugin that exposes the custom SQL functions from [sqlite-fts4](https://github.com/simonw/sqlite-fts4).

  [Interactive demo](https://datasette-sqlite-fts4.datasette.io/24ways-fts4?sql=select%0D%0A++++json_object%28%0D%0A++++++++"label"%2C+articles.title%2C+"href"%2C+articles.url%0D%0A++++%29+as+article%2C%0D%0A++++articles.author%2C%0D%0A++++rank_score%28matchinfo%28articles_fts%2C+"pcx"%29%29+as+score%2C%0D%0A++++rank_bm25%28matchinfo%28articles_fts%2C+"pcnalx"%29%29+as+bm25%2C%0D%0A++++json_object%28%0D%0A++++++++"pre"%2C+annotate_matchinfo%28matchinfo%28articles_fts%2C+"pcxnalyb"%29%2C+"pcxnalyb"%29%0D%0A++++%29+as+annotated_matchinfo%2C%0D%0A++++matchinfo%28articles_fts%2C+"pcxnalyb"%29+as+matchinfo%2C%0D%0A++++decode_matchinfo%28matchinfo%28articles_fts%2C+"pcxnalyb"%29%29+as+decoded_matchinfo%0D%0Afrom%0D%0A++++articles_fts+join+articles+on+articles.rowid+%3D+articles_fts.rowid%0D%0Awhere%0D%0A++++articles_fts+match+%3Asearch%0D%0Aorder+by+bm25&search=jquery+maps). Read [Exploring search relevance algorithms with SQLite](https://simonwillison.net/2019/Jan/7/exploring-search-relevance-algorithms-sqlite/) for further details on this project.

  ## Installation

      pip install datasette-sqlite-fts4

  If you are deploying a database using `datasette publish` you can include this plugin using the `--install` option:

      datasette publish now mydb.db --install=datasette-sqlite-fts4

project_urls:
  {"CI": "https://github.com/simonw/datasette-sqlite-fts4/actions", "Changelog": "https://github.com/simonw/datasette-sqlite-fts4/releases", "Homepage": "https://github.com/simonw/datasette-sqlite-fts4", "Issues": "https://github.com/simonw/datasette-sqlite-fts4/issues"}

requires_dist:
  ["datasette", "sqlite-fts4 (>=1.0.3)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-sqlite-history-json
summary: Track changes to data using triggers and hidden JSON tables
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-sqlite-history-json/
project_url: https://pypi.org/project/datasette-sqlite-history-json/
release_url: https://pypi.org/project/datasette-sqlite-history-json/0.1a0/
requires_dist: ["datasette>=1.0a24", "sqlite-history-json>=0.3a0"]
requires_python: >=3.10
version: 0.1a0
description:
  # datasette-sqlite-history-json

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-history-json.svg)](https://pypi.org/project/datasette-sqlite-history-json/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-sqlite-history-json?include_prereleases&label=changelog)](https://github.com/datasette/datasette-sqlite-history-json/releases)
  [![Tests](https://github.com/datasette/datasette-sqlite-history-json/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-sqlite-history-json/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-sqlite-history-json/blob/main/LICENSE)

  Track changes to data using triggers and hidden JSON tables using [sqlite-history-json](https://github.com/simonw/sqlite-history-json).

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-sqlite-history-json
  ```
  ## Usage

  Install tracking on a table from Datasette's table actions menu (`Enable tracking`) or access the route directly:

  ```bash
  /-/history-json/<database>/<table>/-/enable
  ```

  Once enabled, the table action menu also shows:

  - `View history` → table history page at `/-/history-json/<database>/<table>`
  - `Disable tracking` if tracking is already active

  On each row in a tracked table, the row action menu now shows:

  - `View row history` → `/-/history-json/<database>/<table>/<primary-key>`, with a subtitle showing the number of changes.
  - If the table is not tracked, or the user lacks permission, that action does not appear.

  ## API

  All endpoints return JSON for machine access and can be used in scripts:

  - Table history: `/-/history-json/<database>/<table>.json`
  - Row history: `/-/history-json/<database>/<table>/<primary-key>.json`
  - Filter by operation: `?operation=insert|update|delete`

  ## Permissions

  The plugin registers:

  - `sqlite-history-json` for enabling/disabling tracking (database-level action)
  - `sqlite-history-json-view` for viewing history (table-level action)

  `sqlite-history-json-view` is automatically allowed in non-`--default-deny` mode when the actor can `view-table`.

  Required permissions by feature:

  - Enable tracking: `sqlite-history-json` on `DatabaseResource(database)`
  - Disable tracking: `sqlite-history-json` on `DatabaseResource(database)`
  - View table history page/action: `sqlite-history-json-view` on `TableResource(database, table)` (plus the implicit `view-table` check)
  - View row history action/page: same as table history permission above

  In `--default-deny` mode, `sqlite-history-json-view` is not auto-granted. In that case, provide explicit grants for that action in your permission policy (and keep `sqlite-history-json` grants as needed for managing tracking).

  ## Notes

  `<primary-key>` is URL-safe, tilde-encoded values in the same format Datasette uses internally (for example comma-joined values for composite keys).

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-sqlite-history-json
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-sqlite-history-json/actions", "Changelog": "https://github.com/datasette/datasette-sqlite-history-json/releases", "Homepage": "https://github.com/datasette/datasette-sqlite-history-json", "Issues": "https://github.com/datasette/datasette-sqlite-history-json/issues"}


name: datasette-sqlite-html
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-html
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-html/
project_url: https://pypi.org/project/datasette-sqlite-html/
release_url: https://pypi.org/project/datasette-sqlite-html/0.1.3/
requires_dist: ["datasette", "sqlite-html", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.1.3
project_urls:
  {"CI": "https://github.com/asg017/sqlite-html/actions", "Changelog": "https://github.com/asg017/sqlite-html/releases", "Homepage": "https://github.com/asg017/sqlite-html", "Issues": "https://github.com/asg017/sqlite-html/issues"}


name: datasette-sqlite-http
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-http
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-http/
project_url: https://pypi.org/project/datasette-sqlite-http/
release_url: https://pypi.org/project/datasette-sqlite-http/0.1.1/
requires_dist: ["datasette", "sqlite-http", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.1.1
project_urls:
  {"CI": "https://github.com/asg017/sqlite-http/actions", "Changelog": "https://github.com/asg017/sqlite-http/releases", "Homepage": "https://github.com/asg017/sqlite-http", "Issues": "https://github.com/asg017/sqlite-http/issues"}


name: datasette-sqlite-jsonschema
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-jsonschema
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-jsonschema/
project_url: https://pypi.org/project/datasette-sqlite-jsonschema/
release_url: https://pypi.org/project/datasette-sqlite-jsonschema/0.2.3/
requires_dist: ["datasette", "sqlite-jsonschema", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.2.3
project_urls:
  {"CI": "https://github.com/asg017/sqlite-jsonschema/actions", "Changelog": "https://github.com/asg017/sqlite-jsonschema/releases", "Homepage": "https://github.com/asg017/sqlite-jsonschema", "Issues": "https://github.com/asg017/sqlite-jsonschema/issues"}


name: datasette-sqlite-lines
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-lines
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-lines/
project_url: https://pypi.org/project/datasette-sqlite-lines/
release_url: https://pypi.org/project/datasette-sqlite-lines/0.2.2/
requires_dist: ["datasette", "sqlite-lines", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.2.2
project_urls:
  {"CI": "https://github.com/asg017/sqlite-lines/actions", "Changelog": "https://github.com/asg017/sqlite-lines/releases", "Homepage": "https://github.com/asg017/sqlite-lines", "Issues": "https://github.com/asg017/sqlite-lines/issues"}


name: datasette-sqlite-path
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-path
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-path/
project_url: https://pypi.org/project/datasette-sqlite-path/
release_url: https://pypi.org/project/datasette-sqlite-path/0.2.1/
requires_dist: ["datasette", "sqlite-path", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.2.1
project_urls:
  {"CI": "https://github.com/asg017/sqlite-path/actions", "Changelog": "https://github.com/asg017/sqlite-path/releases", "Homepage": "https://github.com/asg017/sqlite-path", "Issues": "https://github.com/asg017/sqlite-path/issues"}


name: datasette-sqlite-regex
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-regex
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-regex/
project_url: https://pypi.org/project/datasette-sqlite-regex/
release_url: https://pypi.org/project/datasette-sqlite-regex/0.2.3/
requires_dist: ["datasette", "sqlite-regex", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.2.3
project_urls:
  {"CI": "https://github.com/asg017/sqlite-regex/actions", "Changelog": "https://github.com/asg017/sqlite-regex/releases", "Homepage": "https://github.com/asg017/sqlite-regex", "Issues": "https://github.com/asg017/sqlite-regex/issues"}


name: datasette-sqlite-trace
summary: Datasette plugin that prints all executed SQL to stderr
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-sqlite-trace/
project_url: https://pypi.org/project/datasette-sqlite-trace/
release_url: https://pypi.org/project/datasette-sqlite-trace/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
version: 0.1
description:
  # datasette-sqlite-trace

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-trace.svg)](https://pypi.org/project/datasette-sqlite-trace/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-sqlite-trace?include_prereleases&label=changelog)](https://github.com/datasette/datasette-sqlite-trace/releases)
  [![Tests](https://github.com/datasette/datasette-sqlite-trace/workflows/Test/badge.svg)](https://github.com/datasette/datasette-sqlite-trace/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-sqlite-trace/blob/main/LICENSE)

  Datasette plugin that prints all executed SQL to stderr

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-sqlite-trace
  ```

  ## Usage

  Once installed, this plugin will log every SQL query executed by Datasette to standard error.

project_urls:
  {"CI": "https://github.com/datasette/datasette-sqlite-trace/actions", "Changelog": "https://github.com/datasette/datasette-sqlite-trace/releases", "Homepage": "https://github.com/datasette/datasette-sqlite-trace", "Issues": "https://github.com/datasette/datasette-sqlite-trace/issues"}


name: datasette-sqlite-ulid
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-ulid
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-ulid/
project_url: https://pypi.org/project/datasette-sqlite-ulid/
release_url: https://pypi.org/project/datasette-sqlite-ulid/0.2.1/
requires_dist: ["datasette", "sqlite-ulid", "pytest ; extra == 'test'"]
requires_python: >=3.6
version: 0.2.1
project_urls:
  {"CI": "https://github.com/asg017/sqlite-ulid/actions", "Changelog": "https://github.com/asg017/sqlite-ulid/releases", "Homepage": "https://github.com/asg017/sqlite-ulid", "Issues": "https://github.com/asg017/sqlite-ulid/issues"}


name: datasette-sqlite-url-lite
summary: A pure Python alternative to sqlite-url ready to be used in Datasette Lite
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-sqlite-url-lite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-url-lite/
project_url: https://pypi.org/project/datasette-sqlite-url-lite/
release_url: https://pypi.org/project/datasette-sqlite-url-lite/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-sqlite-url-lite

  [![PyPI](https://img.shields.io/pypi/v/datasette-sqlite-url-lite.svg)](https://pypi.org/project/datasette-sqlite-url-lite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-sqlite-url-lite?include_prereleases&label=changelog)](https://github.com/simonw/datasette-sqlite-url-lite/releases)
  [![Tests](https://github.com/simonw/datasette-sqlite-url-lite/workflows/Test/badge.svg)](https://github.com/simonw/datasette-sqlite-url-lite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-sqlite-url-lite/blob/main/LICENSE)

  A pure Python alternative to [sqlite-url](https://github.com/asg017/sqlite-url) ready to be used in [Datasette Lite](https://lite.datasette.io/)

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-sqlite-url-lite

  Or add it to Datasette Lite with `https://lite.datasette.io/?install=datasette-sqlite-url-lite`.

  ## Usage

  This plugin adds the following SQL functions:

  ```sql
  select url_valid('https://sqlite.org'); -- 1
  select url_scheme('https://www.sqlite.org/vtab.html#usage'); -- 'https'
  select url_host('https://www.sqlite.org/vtab.html#usage'); -- 'www.sqlite.org'
  select url_path('https://www.sqlite.org/vtab.html#usage'); -- '/vtab.html'
  select url_fragment('https://www.sqlite.org/vtab.html#usage'); -- 'usage'
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-sqlite-url-lite
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-sqlite-url-lite/actions", "Changelog": "https://github.com/simonw/datasette-sqlite-url-lite/releases", "Homepage": "https://github.com/simonw/datasette-sqlite-url-lite", "Issues": "https://github.com/simonw/datasette-sqlite-url-lite/issues"}


name: datasette-sqlite-url
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-url
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-url/
project_url: https://pypi.org/project/datasette-sqlite-url/
release_url: https://pypi.org/project/datasette-sqlite-url/0.1.0/
requires_dist: ["datasette", "sqlite-url", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.1.0
project_urls:
  {"CI": "https://github.com/asg017/sqlite-url/actions", "Changelog": "https://github.com/asg017/sqlite-url/releases", "Homepage": "https://github.com/asg017/sqlite-url", "Issues": "https://github.com/asg017/sqlite-url/issues"}


name: datasette-sqlite-vss
classifiers: []
author: Alex Garcia
description_content_type: text/markdown
home_page: https://github.com/asg017/sqlite-vss
license: MIT License, Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-sqlite-vss/
project_url: https://pypi.org/project/datasette-sqlite-vss/
release_url: https://pypi.org/project/datasette-sqlite-vss/0.1.2/
requires_dist: ["datasette", "sqlite-vss", "pytest ; extra == 'test'"]
requires_python: >=3.7
version: 0.1.2
project_urls:
  {"CI": "https://github.com/asg017/sqlite-vss/actions", "Changelog": "https://github.com/asg017/sqlite-vss/releases", "Homepage": "https://github.com/asg017/sqlite-vss", "Issues": "https://github.com/asg017/sqlite-vss/issues"}


name: datasette-statistics
summary: SQL statistics functions for Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-statistics/
project_url: https://pypi.org/project/datasette-statistics/
release_url: https://pypi.org/project/datasette-statistics/0.2.1/
requires_python: >=3.6
version: 0.2.1
description:
  # datasette-statistics

  [![PyPI](https://img.shields.io/pypi/v/datasette-statistics.svg)](https://pypi.org/project/datasette-statistics/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-statistics?include_prereleases&label=changelog)](https://github.com/simonw/datasette-statistics/releases)
  [![Tests](https://github.com/simonw/datasette-statistics/workflows/Test/badge.svg)](https://github.com/simonw/datasette-statistics/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-statistics/blob/main/LICENSE)

  SQL statistics functions for Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-statistics
  ```
  ## Usage

  This plugin adds new SQL aggregate functions for use within Datasette:

  - `statistics_mean()` for calculating the [mean](https://docs.python.org/3/library/statistics.html#statistics.mean)
  - `statistics_geometric_mean()` for calculating the [geometric mean](https://docs.python.org/3/library/statistics.html#statistics.geometric_mean) (requires Python 3.8+)
  - `statistics_median()` for calculating the [median](https://docs.python.org/3/library/statistics.html#statistics.median)
  - `statistics_median_low()` for calculating the [low median](https://docs.python.org/3/library/statistics.html#statistics.median_low)
  - `statistics_median_high()` for calculating the [high median](https://docs.python.org/3/library/statistics.html#statistics.median_high)
  - `statistics_mode()` for calculating the [mode](https://docs.python.org/3/library/statistics.html#statistics.mode)
  - `statistics_stdev()` for calculating the [sample standard deviation](https://docs.python.org/3/library/statistics.html#statistics.stdev)
  - `statistics_pstdev()` for calculating the [population standard deviation](https://docs.python.org/3/library/statistics.html#statistics.pstdev)
  - `statistics_variance()` for calculating the [sample variance](https://docs.python.org/3/library/statistics.html#statistics.variance)
  - `statistics_pvariance()` for calculating the [population variance](https://docs.python.org/3/library/statistics.html#statistics.pvariance)

  These all use the implementations from the [Python statistics library](https://docs.python.org/3/library/statistics.html).

  Use them like this:
  ```sql
  select statistics_mean(numeric_column) from mytable
  ```
  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-statistics
  python3 -mvenv venv
  source venv/bin/activate
  ```
  Or if you are using `pipenv`:
  ```bash
  pipenv shell
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-statistics/actions", "Changelog": "https://github.com/simonw/datasette-statistics/releases", "Homepage": "https://github.com/simonw/datasette-statistics", "Issues": "https://github.com/simonw/datasette-statistics/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-mock ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-studio
summary: Datasette pre-configured with useful plugins
classifiers: ["License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-studio/
project_url: https://pypi.org/project/datasette-studio/
release_url: https://pypi.org/project/datasette-studio/0.1a4/
requires_python: >=3.8
version: 0.1a4
description:
  # datasette-studio

  [![PyPI](https://img.shields.io/pypi/v/datasette-studio.svg)](https://pypi.org/project/datasette-studio/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-studio?include_prereleases&label=changelog)](https://github.com/datasette/datasette-studio/releases)
  [![Tests](https://github.com/datasette/datasette-studio/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-studio/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-studio/blob/master/LICENSE)

  Datasette pre-configured with useful plugins.

  **Experimental alpha release**. This is an early experiment at the moment.

  [Try this out in GitHub Codespaces](https://github.com/codespaces/new?repo=datasette/studio)

  ## Installation

  This tool makes Datasette (currently the 1.0 alpha series) available as `datasette-studio` with a set of useful plugins pre-installed.

  It is _strongly_ recommended to install this using [pipx](https://pipx.pypa.io/), since doing so will ensure that this version of Datasette has its own isolated environment.

  ```bash
  pipx install datasette-studio
  ```

  ## Usage

  For help, run:
  ```bash
  datasette-studio --help
  ```
  To list available plugins run:
  ```bash
  datasette-studio plugins
  ```
  To install additional plugins use:
  ```bash
  datasette-studio install datasette-graphql
  ```
  The `datasette-studio` command is an alias for regular `datasette`, so [consult the Datasette documentation](https://docs.datasette.io/en/latest/) for more information on how to use it.

project_urls:
  {"CI": "https://github.com/datasette/datasette-studio/actions", "Changelog": "https://github.com/datasette/datasette-studio/releases", "Homepage": "https://github.com/datasette/datasette-studio", "Issues": "https://github.com/datasette/datasette-studio/issues"}

requires_dist:
  ["datasette>=1.0a13", "datasette-edit-schema", "datasette-write-ui", "datasette-configure-fts", "datasette-write", "datasette-upload-csvs", "datasette-enrichments", "datasette-enrichments-quickjs", "datasette-enrichments-re2", "datasette-enrichments-jinja", "datasette-copyable", "datasette-export-database", "datasette-enrichments-gpt", "datasette-import", "datasette-extract", "datasette-secrets"]


name: datasette-table-diagram
summary: Show Entity Relationship diagrams of tables in Datasette
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-table-diagram/
project_url: https://pypi.org/project/datasette-table-diagram/
release_url: https://pypi.org/project/datasette-table-diagram/0.1a0/
requires_dist: ["datasette>=1.0a25"]
requires_python: >=3.10
version: 0.1a0
description:
  # datasette-table-diagram

  [![PyPI](https://img.shields.io/pypi/v/datasette-table-diagram.svg)](https://pypi.org/project/datasette-table-diagram/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-table-diagram?include_prereleases&label=changelog)](https://github.com/datasette/datasette-table-diagram/releases)
  [![Tests](https://github.com/datasette/datasette-table-diagram/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-table-diagram/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-table-diagram/blob/main/LICENSE)

  Show Entity Relationship diagrams of tables in Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-table-diagram
  ```
  ## Usage

  Databases gain a new database action menu item called "Table diagram", which links to a page that displays Mermaid Entity Relationship diagrams of the tables in that database.

  These diagrams can be panned and zoomed, or users can click a table name to zoom to that table and its relationships.

  Users will only see tables that they have permission to view.

  ![Screenshot of a diagram of the fixtures.db table - tables are joined by curved lines, there is a + / - / reset button and a list of tables.](https://raw.githubusercontent.com/datasette/datasette-table-diagram/refs/heads/main/screenshot.png)

  ## Development

  To set up this plugin locally, first checkout the code. You can confirm it is available like this:
  ```bash
  cd datasette-table-diagram
  # Confirm the plugin is visible
  uv run datasette plugins
  ```
  To run the tests:
  ```bash
  uv run pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-table-diagram/actions", "Changelog": "https://github.com/datasette/datasette-table-diagram/releases", "Homepage": "https://github.com/datasette/datasette-table-diagram", "Issues": "https://github.com/datasette/datasette-table-diagram/issues"}


name: datasette-tail
summary: Tools for tailing your database
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-tail/
project_url: https://pypi.org/project/datasette-tail/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-tail/0.1a2/
requires_python: >=3.10
version: 0.1a2
description:
  # datasette-tail

  [![PyPI](https://img.shields.io/pypi/v/datasette-tail.svg)](https://pypi.org/project/datasette-tail/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-tail?include_prereleases&label=changelog)](https://github.com/datasette/datasette-tail/releases)
  [![Tests](https://github.com/datasette/datasette-tail/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-tail/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-tail/blob/main/LICENSE)

  Tools for tailing your database

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-tail
  ```
  ## Usage

  This plugin provides a simple debugging tool. Visit `/dbname/-/tail` to see a JSON representation of changes to your database. The first time you visit that page it will represent everything in the specified database. Refreshing the page will show any changes since the last time you loaded that page.

  Use `/dbname/-/tail.json` to get the raw JSON.

  The plugin obeys the `view-database` permission.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-tail
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-tail/actions", "Changelog": "https://github.com/datasette/datasette-tail/releases", "Homepage": "https://github.com/datasette/datasette-tail", "Issues": "https://github.com/datasette/datasette-tail/issues"}

requires_dist:
  ["datasette>=1.0a20", "dictdiffer", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]


name: datasette-template-request
summary: Expose the Datasette request object to custom templates
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-template-request
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-template-request/
project_url: https://pypi.org/project/datasette-template-request/
release_url: https://pypi.org/project/datasette-template-request/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1
description:
  # datasette-template-request

  [![PyPI](https://img.shields.io/pypi/v/datasette-template-request.svg)](https://pypi.org/project/datasette-template-request/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-template-request?include_prereleases&label=changelog)](https://github.com/simonw/datasette-template-request/releases)
  [![Tests](https://github.com/simonw/datasette-template-request/workflows/Test/badge.svg)](https://github.com/simonw/datasette-template-request/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-template-request/blob/main/LICENSE)

  Expose the Datasette request object to custom templates

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-template-request

  ## Usage

  Once this plugin is installed, Datasette [custom templates](https://docs.datasette.io/en/stable/custom_templates.html) can use `{{ request }}` to access the current [request object](https://docs.datasette.io/en/stable/internals.html#request-object). For example, to access `?name=Cleo` in the query string a template could use this:

      Name: {{ request.args.name }}

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-template-request
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-template-request/actions", "Changelog": "https://github.com/simonw/datasette-template-request/releases", "Homepage": "https://github.com/simonw/datasette-template-request", "Issues": "https://github.com/simonw/datasette-template-request/issues"}


name: datasette-template-sql
summary: Datasette plugin for executing SQL queries from templates
classifiers: []
author: Simon Willison
description_content_type: text/markdown
license_expression: Apache-2.0
package_url: https://pypi.org/project/datasette-template-sql/
project_url: https://pypi.org/project/datasette-template-sql/
release_url: https://pypi.org/project/datasette-template-sql/1.0.3/
requires_dist: ["datasette>=0.54"]
requires_python: >=3.10
version: 1.0.3
description:
  # datasette-template-sql

  [![PyPI](https://img.shields.io/pypi/v/datasette-template-sql.svg)](https://pypi.org/project/datasette-template-sql/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-template-sql?include_prereleases&label=changelog)](https://github.com/simonw/datasette-template-sql/releases)
  [![Tests](https://github.com/simonw/datasette-template-sql/workflows/Test/badge.svg)](https://github.com/simonw/datasette-template-sql/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-template-sql/blob/main/LICENSE)

  Datasette plugin for executing SQL queries from templates.

  ## Examples

  [datasette.io](https://datasette.io/) uses this plugin extensively with [custom page templates](https://docs.datasette.io/en/stable/custom_templates.html#custom-pages), check out [simonw/datasette.io](https://github.com/simonw/datasette.io) to see how it works.

  [www.niche-museums.com](https://www.niche-museums.com/) uses this plugin to run a custom themed website on top of Datasette. The full source code for the site [is here](https://github.com/simonw/museums) - see also [niche-museums.com, powered by Datasette](https://simonwillison.net/2019/Nov/25/niche-museums/).

  [simonw/til](https://github.com/simonw/til) is another simple example, described in [Using a self-rewriting README powered by GitHub Actions to track TILs](https://simonwillison.net/2020/Apr/20/self-rewriting-readme/).

  ## Installation

  Run this command to install the plugin in the same environment as Datasette:

      $ pip install datasette-template-sql

  ## Usage

  This plugin makes a new function, `sql(sql_query)`, available to your Datasette templates.

  You can use it like this:

  ```html+jinja
  {% for row in sql("select 1 + 1 as two, 2 * 4 as eight") %}
      {% for key in row.keys() %}
          {{ key }}: {{ row[key] }}<br>
      {% endfor %}
  {% endfor %}
  ```

  The plugin will execute SQL against the current database for the page in  `database.html`, `table.html` and `row.html` templates. If a template does not have a current database (`index.html` for example) the query will execute against the first attached database.

  ### Queries with arguments

  You can construct a SQL query using `?` or `:name` parameter syntax by passing a list or dictionary as a second argument:

  ```html+jinja
  {% for row in sql("select distinct topic from til order by topic") %}
      <h2>{{ row.topic }}</h2>
      <ul>
          {% for til in sql("select * from til where topic = ?", [row.topic]) %}
              <li><a href="{{ til.url }}">{{ til.title }}</a> - {{ til.created[:10] }}</li>
          {% endfor %}
      </ul>
  {% endfor %}
  ```

  Here's the same example using the `:topic` style of parameters:

  ```html+jinja
  {% for row in sql("select distinct topic from til order by topic") %}
      <h2>{{ row.topic }}</h2>
      <ul>
          {% for til in sql("select * from til where topic = :topic", {"topic": row.topic}) %}
              <li><a href="{{ til.url }}">{{ til.title }}</a> - {{ til.created[:10] }}</li>
          {% endfor %}
      </ul>
  {% endfor %}
  ```

  ### Querying a different database

  You can pass an optional `database=` argument to specify a named database to use for the query. For example, if you have attached a `news.db` database you could use this:

  ```html+jinja
  {% for article in sql(
      "select headline, date, summary from articles order by date desc limit 5",
      database="news"
  ) %}
      <h3>{{ article.headline }}</h2>
      <p class="date">{{ article.date }}</p>
      <p>{{ article.summary }}</p>
  {% endfor %}
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-template-sql/actions", "Changelog": "https://github.com/simonw/datasette-template-sql/releases", "Homepage": "https://github.com/simonw/datasette-template-sql", "Issues": "https://github.com/simonw/datasette-template-sql/issues"}


name: datasette-test-plugin
summary: Part of datasette-test
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-test-plugin/
project_url: https://pypi.org/project/datasette-test-plugin/
release_url: https://pypi.org/project/datasette-test-plugin/0.1/
requires_dist: ["datasette", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]
requires_python: >=3.8
version: 0.1
description:
  # datasette-test-plugin

  [![PyPI](https://img.shields.io/pypi/v/datasette-test-plugin.svg)](https://pypi.org/project/datasette-test-plugin/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-test-plugin?include_prereleases&label=changelog)](https://github.com/datasette/datasette-test-plugin/releases)
  [![Tests](https://github.com/datasette/datasette-test-plugin/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-test-plugin/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-test-plugin/blob/main/LICENSE)

  Part of [datasette-test](https://github.com/datasette/datasette-test).

project_urls:
  {"CI": "https://github.com/datasette/datasette-test-plugin/actions", "Changelog": "https://github.com/datasette/datasette-test-plugin/releases", "Homepage": "https://github.com/datasette/datasette-test-plugin", "Issues": "https://github.com/datasette/datasette-test-plugin/issues"}


name: datasette-test
summary: Utilities to help write tests for Datasette plugins and applications
classifiers: ["License :: OSI Approved :: Apache Software License"]
author: Datasette
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-test/
project_url: https://pypi.org/project/datasette-test/
release_url: https://pypi.org/project/datasette-test/0.3.2/
requires_python: >=3.8
version: 0.3.2
description:
  # datasette-test

  [![PyPI](https://img.shields.io/pypi/v/datasette-test.svg)](https://pypi.org/project/datasette-test/)
  [![Tests](https://github.com/datasette/datasette-test/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-test/actions/workflows/test.yml)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-test?include_prereleases&label=changelog)](https://github.com/datasette/datasette-test/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-test/blob/main/LICENSE)

  Utilities to help write tests for Datasette plugins and applications

  ## Installation

  Install this library using `pip`:
  ```bash
  pip install datasette-test
  ```
  ## Tests that use plugin configuration

  Datasette 1.0a8 enforced a configuration change where plugins are no longer configured in metadata, but instead use a configuration file.

  This can result in test failures in projects that use the `Datasette(metadata={"plugins": {"...": "..."}})` pattern to test out plugin configuration.

  You can solve this using `datasette_test.Datasette`, a subclass that works with Datasette versions both before and after this breaking change:

  ```python
  from datasette_test import Datasette
  import pytest

  @pytest.mark.asyncio
  async def test_datasette():
      ds = Datasette(plugin_config={"my-plugin": {"config": "goes here"})
  ```
  This subclass detects if the underlying plugin needs configuration in metadata or config and instantiates the class correctly either way.

  You can also use this class while continuing to pass `metadata={"plugins": ...}` - the class will move that configuration to config when necessary.

  ## permissions= convenience argument

  Datasette 1.0a introduces a [more convenient way](https://docs.datasette.io/en/1.0a13/authentication.html#other-permissions-in-datasette-yaml) of defining permissions directly in the configuration:
  ```python
  ds = Datasette(config={"permissions": {"view-instance": {"id": "root"}}})
  ```
  This is not supported by Datasette pre 1.0 - but you can use the `permissions=` argument in `datasette_test.Datasette` to achieve the same effect:
  ```python
  ds = Datasette(permissions={"view-instance": {"id": "root"}})
  ```
  This will work across both major Datasette versions.

  ## wait_until_responds(url, timeout=5.0)

  Some Datasette plugin test suites launch a Datasette server and then need to wait for that server to become available before continuing.

  Call this function to wait until the server becomes available, or raise an error if it takes longer than the timeout:

  ```python
  from datasette_test import wait_until_responds

  def test_server():
      # ... start server ...
      wait_until_responds("http://localhost:8001")
      # Now run tests
  ```

  ## actor_cookie(datasette, actor)

  Equivalent to `datasette.client.actor_cookie()` in Datasette 1.0a+. Example usage:

  ```python
  @pytest.mark.asyncio
  async def test_permissions():
      ds = Datasette(permissions={"view-instance": {"id": "root"}})
      response = await ds.client.get("/")
      assert response.status_code == 403
      response = await ds.client.get(
          "/", cookies={"ds_actor": actor_cookie(ds, {"id": "root"})}
      )
      assert response.status_code == 200
  ```

  ## Development

  To contribute to this library, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-test
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-test/actions", "Changelog": "https://github.com/datasette/datasette-test/releases", "Homepage": "https://github.com/datasette/datasette-test", "Issues": "https://github.com/datasette/datasette-test/issues"}

requires_dist:
  ["datasette", "httpx", "packaging", "datasette-test-plugin", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]


name: datasette-tiddlywiki
summary: Run TiddlyWiki in Datasette and save Tiddlers to a SQLite database
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-tiddlywiki
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-tiddlywiki/
project_url: https://pypi.org/project/datasette-tiddlywiki/
release_url: https://pypi.org/project/datasette-tiddlywiki/0.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.2
description:
  # datasette-tiddlywiki

  [![PyPI](https://img.shields.io/pypi/v/datasette-tiddlywiki.svg)](https://pypi.org/project/datasette-tiddlywiki/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-tiddlywiki?include_prereleases&label=changelog)](https://github.com/simonw/datasette-tiddlywiki/releases)
  [![Tests](https://github.com/simonw/datasette-tiddlywiki/workflows/Test/badge.svg)](https://github.com/simonw/datasette-tiddlywiki/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-tiddlywiki/blob/main/LICENSE)

  Run [TiddlyWiki](https://tiddlywiki.com/) in Datasette and save Tiddlers to a SQLite database

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-tiddlywiki

  ## Usage

  Start Datasette with a `tiddlywiki.db` database. You can create it if it does not yet exist using `--create`.

  You need to be signed in as the `root` user to write to the wiki, so use the `--root` option and click on the link it provides:

      % datasette tiddlywiki.db --create --root
      http://127.0.0.1:8001/-/auth-token?token=456670f1e8d01a8a33b71e17653130de17387336e29afcdfb4ab3d18261e6630
      # ...

  Navigate to `/-/tiddlywiki` on your instance to interact with TiddlyWiki.

  ## Authentication and permissions

  By default, the wiki can be read by anyone who has permission to read the `tiddlywiki.db` database. Only the signed in `root` user can write to it.

  You can sign in using the `--root` option described above, or you can set a password for that user using the [datasette-auth-passwords](https://datasette.io/plugins/datasette-auth-passwords) plugin and sign in using the `/-/login` page.

  You can use the `edit-tiddlywiki` permission to grant edit permisions to other users, using another plugin such as [datasette-permissions-sql](https://datasette.io/plugins/datasette-permissions-sql).

  You can use the `view-database` permission against the `tiddlywiki` database to control who can view the wiki.

  Datasette's permissions mechanism is described in full in [the Datasette documentation](https://docs.datasette.io/en/stable/authentication.html).

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-tiddlywiki
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-tiddlywiki/actions", "Changelog": "https://github.com/simonw/datasette-tiddlywiki/releases", "Homepage": "https://github.com/simonw/datasette-tiddlywiki", "Issues": "https://github.com/simonw/datasette-tiddlywiki/issues"}


name: datasette-tiles
summary: Mapping tile server for Datasette, serving tiles from MBTiles packages
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-tiles
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-tiles/
project_url: https://pypi.org/project/datasette-tiles/
release_url: https://pypi.org/project/datasette-tiles/0.6.1/
requires_python: >=3.6
version: 0.6.1
description:
  # datasette-tiles

  [![PyPI](https://img.shields.io/pypi/v/datasette-tiles.svg)](https://pypi.org/project/datasette-tiles/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-tiles?include_prereleases&label=changelog)](https://github.com/simonw/datasette-tiles/releases)
  [![Tests](https://github.com/simonw/datasette-tiles/workflows/Test/badge.svg)](https://github.com/simonw/datasette-tiles/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-tiles/blob/main/LICENSE)

  Datasette plugin for serving MBTiles map tiles

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-tiles

  ## Demo

  You can try this plugin out at https://datasette-tiles-demo.datasette.io/-/tiles

  ## Usage

  This plugin scans all database files connected to Datasette to see if any of them are valid MBTiles databases.

  It can then serve tiles from those databases at the following URL:

      /-/tiles/db-name/zoom/x/y.png

  An example map for each database demonstrating the configured minimum and maximum zoom for that database can be found at `/-/tiles/db-name` - this can also be accessed via the table and database action menus for that database.

  Visit `/-/tiles` for an index page of attached valid databases.

  You can install the [datasette-basemap](https://datasette.io/plugins/datasette-basemap) plugin to get a `basemap` default set of tiles, handling zoom levels 0 to 6 using OpenStreetMap.

  ### Tile coordinate systems

  There are two tile coordinate systems in common use for online maps. The first is used by OpenStreetMap and Google Maps, the second is from a specification called [Tile Map Service](https://en.wikipedia.org/wiki/Tile_Map_Service), or TMS.

  Both systems use three components: `z/x/y` - where `z` is the zoom level, `x` is the column and `y` is the row.

  The difference is in the way the `y` value is counted. OpenStreetMap has y=0 at the top. TMS has y=0 at the bottom.

  An illustrative example: at zoom level 2 the map is divided into 16 total tiles. The OpenStreetMap scheme numbers them like so:

      0/0  1/0  2/0  3/0
      0/1  1/1  2/1  3/1
      0/2  1/2  2/2  3/2
      0/3  1/3  2/3  3/3

  The TMS scheme looks like this:

      0/3  1/3  2/3  3/3
      0/2  1/2  2/2  3/2
      0/1  1/1  2/1  3/1
      0/0  1/0  2/0  3/0

  `datasette-tiles` can serve tiles using either of these standards. For the OpenStreetMap / Google Maps 0-at-the-top system, use the following URL:

      /-/tiles/database-name/{z}/{x}/{y}.png

  For the TMS 0-at-the-bottom system, use this:

      /-/tiles-tms/database-name/{z}/{x}/{y}.png

  ### Configuring a Leaflet tile layer

  The following JavaScript will configure a [Leaflet TileLayer](https://leafletjs.com/reference-1.7.1.html#tilelayer) for use with this plugin:

  ```javascript
  var tiles = leaflet.tileLayer("/-/tiles/basemap/{z}/{x}/{y}.png", {
    minZoom: 0,
    maxZoom: 6,
    attribution: "\u00a9 OpenStreetMap contributors"
  });
  ```

  ### Tile stacks

  `datasette-tiles` can be configured to serve tiles from multiple attached MBTiles files, searching each database in order for a tile and falling back to the next in line if that tile is not found.

  For a demo of this in action, visit https://datasette-tiles-demo.datasette.io/-/tiles-stack and zoom in on Japan. It should start showing [Stamen's Toner map](maps.stamen.com) of Japan once you get to zoom level 6 and 7.

  The `/-/tiles-stack/{z}/{x}/{y}.png` endpoint provides this feature.

  If you start Datasette like this:

      datasette world.mbtiles country.mbtiles city1.mbtiles city2.mbtiles

  Any requests for a tile from the `/-/tiles-stack` path will first check the `city2` database, than `city1`, then `country`, then `world`.

  If you have the [datasette-basemap](https://datasette.io/plugins/datasette-basemap) plugin installed it will be given special treatment: the `basemap` database will always be the last database checked for a tile.

  Rather than rely on the order in which databases were attached, you can instead configure an explicit order using the `tiles-stack-order` plugin setting. Add the following to your `metadata.json` file:

  ```json
  {
      "plugins": {
          "datasette-tiles": {
              "tiles-stack-order": ["world", "country"]
          }
      }
  }
  ```

  You can then run Datasette like this:

      datasette -m metadata.json country.mbtiles world.mbtiles

  This endpoint serves tiles using the OpenStreetMap / Google Maps coordinate system. To load tiles using the TMS coordinate system use this endpoint instead:

      /-/tiles-stack-tms/{z}/{x}/{y}.png

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-tiles
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-tiles/actions", "Changelpog": "https://github.com/simonw/datasette-tiles/releases", "Homepage": "https://github.com/simonw/datasette-tiles", "Issues": "https://github.com/simonw/datasette-tiles/issues"}

requires_dist:
  ["datasette", "datasette-leaflet (>=0.2.2)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "datasette-basemap (>=0.2) ; extra == 'test'"]


name: datasette-total-page-time
summary: Add a note to the Datasette footer measuring the total page load time
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-total-page-time
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-total-page-time/
project_url: https://pypi.org/project/datasette-total-page-time/
release_url: https://pypi.org/project/datasette-total-page-time/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-total-page-time

  [![PyPI](https://img.shields.io/pypi/v/datasette-total-page-time.svg)](https://pypi.org/project/datasette-total-page-time/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-total-page-time?include_prereleases&label=changelog)](https://github.com/simonw/datasette-total-page-time/releases)
  [![Tests](https://github.com/simonw/datasette-total-page-time/workflows/Test/badge.svg)](https://github.com/simonw/datasette-total-page-time/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-total-page-time/blob/main/LICENSE)

  Add a note to the Datasette footer measuring the total page load time

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-total-page-time

  ## Usage

  Once this plugin is installed, a note will appear in the footer of every page showing how long the page took to generate.

  > Queries took 326.74ms · Page took 386.310ms

  ## How it works

  Measuring how long a page takes to load and then injecting that note into the page is tricky, because you need to finish generating the page before you know how long it took to load it!

  This plugin uses the [asgi_wrapper](https://docs.datasette.io/en/stable/plugin_hooks.html#asgi-wrapper-datasette) plugin hook to measure the time taken by Datasette and then inject the following JavaScript at the bottom of the response, after the closing `</html>` tag but with the correct measured value:

  ```html
  <script>
  let footer = document.querySelector("footer");
  if (footer) {
      let ms = 37.224;
      let s = ` &middot; Page took ${ms.toFixed(3)}ms`;
      footer.innerHTML += s;
  }
  </script>
  ```
  This script is injected only on pages with the `text/html` content type - so it should not affect JSON or CSV returned by Datasette.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-total-page-time
      python3 -mvenv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-total-page-time/actions", "Changelog": "https://github.com/simonw/datasette-total-page-time/releases", "Homepage": "https://github.com/simonw/datasette-total-page-time", "Issues": "https://github.com/simonw/datasette-total-page-time/issues"}


name: datasette-transactions
summary: API for executing multiple queries within a transaction
classifiers: ["Framework :: Datasette"]
author: Datasette
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-transactions/
project_url: https://pypi.org/project/datasette-transactions/
release_url: https://pypi.org/project/datasette-transactions/0.1a0/
requires_dist: ["datasette>=1.0a22", "httpx>=0.25"]
requires_python: >=3.10
version: 0.1a0
description:
  # datasette-transactions

  [![PyPI](https://img.shields.io/pypi/v/datasette-transactions.svg)](https://pypi.org/project/datasette-transactions/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-transactions?include_prereleases&label=changelog)](https://github.com/datasette/datasette-transactions/releases)
  [![Tests](https://github.com/datasette/datasette-transactions/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-transactions/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-transactions/blob/main/LICENSE)

  A Datasette plugin providing an API for executing multiple SQL commands within a single transaction, with support for SQLite authorization callbacks to control table-level read/write access.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-transactions
  ```

  ## Usage

  ### Begin a Transaction

  ```http
  POST /-/transactions/begin/<database>
  Content-Type: application/json

  {
    "read": ["table1", "table2"],
    "write": ["table1"],
    "timeout_ms": 5000
  }
  ```

  **Parameters:**
  - `read`: List of tables the transaction can read from
  - `write`: List of tables the transaction can write to
  - `timeout_ms`: Optional timeout in milliseconds (transaction auto-rolls back after this time)

  **Response:**
  ```json
  {
    "ok": true,
    "transaction_id": "550e8400-e29b-41d4-a716-446655440000"
  }
  ```

  **Required Permissions:**
  - `execute-sql` on the database
  - `view-table` on each table in the `read` list
  - `insert-row` and `update-row` on each table in the `write` list

  ### Execute SQL

  ```http
  POST /-/transactions/<transaction_id>
  Content-Type: application/json

  {
    "sql": "INSERT INTO table1 (name) VALUES (:name)",
    "params": {"name": "Alice"}
  }
  ```

  **Response:**
  ```json
  {
    "ok": true,
    "rows": [],
    "columns": [],
    "truncated": false
  }
  ```

  For SELECT queries, `rows` contains the results as a list of objects.

  ### Create a Savepoint

  ```http
  POST /-/transactions/<transaction_id>/savepoint
  Content-Type: application/json

  {
    "name": "before_batch"
  }
  ```

  **Response:**
  ```json
  {
    "ok": true,
    "savepoint": "before_batch"
  }
  ```

  ### Release a Savepoint

  ```http
  POST /-/transactions/<transaction_id>/release
  Content-Type: application/json

  {
    "name": "before_batch"
  }
  ```

  ### Rollback to a Savepoint

  ```http
  POST /-/transactions/<transaction_id>/rollback-to
  Content-Type: application/json

  {
    "name": "before_batch"
  }
  ```

  ### Commit a Transaction

  ```http
  POST /-/transactions/commit/<transaction_id>
  ```

  **Response:**
  ```json
  {
    "ok": true
  }
  ```

  ### Rollback a Transaction

  ```http
  POST /-/transactions/rollback/<transaction_id>
  ```

  **Response:**
  ```json
  {
    "ok": true
  }
  ```

  ## Error Codes

  | Status | Meaning |
  |--------|---------|
  | 400 | Bad request (invalid JSON, missing parameters, SQL error) |
  | 403 | Permission denied (missing required permissions, table access denied) |
  | 404 | Transaction or savepoint not found |
  | 405 | Method not allowed (use POST) |
  | 410 | Transaction expired (timed out) |
  | 429 | Too many concurrent transactions (max 5 per database) |

  ## Authorization

  The plugin uses SQLite's `set_authorizer()` callback to enforce table-level permissions:

  - Reading from tables not in `read` list is blocked
  - Writing to tables not in `write` list is blocked
  - `ATTACH DATABASE` is always blocked

  ## Concurrency

  - Maximum 5 concurrent transactions per database
  - Read-only transactions use `BEGIN DEFERRED` (allows concurrent reads)
  - Write transactions use `BEGIN IMMEDIATE` (exclusive lock)

  ## Example: Batch Insert with Rollback on Error

  ```python
  import httpx

  # Begin transaction
  response = httpx.post(
      "http://localhost:8001/-/transactions/begin/mydb",
      json={"read": ["users"], "write": ["users"]}
  )
  tx_id = response.json()["transaction_id"]

  try:
      # Insert multiple rows
      for user in users:
          httpx.post(
              f"http://localhost:8001/-/transactions/{tx_id}",
              json={
                  "sql": "INSERT INTO users (name, email) VALUES (:name, :email)",
                  "params": user
              }
          )

      # Commit if all succeeded
      httpx.post(f"http://localhost:8001/-/transactions/commit/{tx_id}")
  except Exception:
      # Rollback on error
      httpx.post(f"http://localhost:8001/-/transactions/rollback/{tx_id}")
      raise
  ```

  ## Development

  To set up this plugin locally, first checkout the code:
  ```bash
  cd datasette-transactions
  uv run pytest  # Run tests
  ```

  This project follows TDD (Test-Driven Development). See `CLAUDE.md` for development guidelines.

project_urls:
  {"CI": "https://github.com/datasette/datasette-transactions/actions", "Changelog": "https://github.com/datasette/datasette-transactions/releases", "Homepage": "https://github.com/datasette/datasette-transactions", "Issues": "https://github.com/datasette/datasette-transactions/issues"}


name: datasette-turnstile
summary: Integrate CAPTCHAs powered by Cloudflare Turnstile
classifiers: ["Framework :: Datasette"]
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-turnstile/
project_url: https://pypi.org/project/datasette-turnstile/
release_url: https://pypi.org/project/datasette-turnstile/0.1a3/
requires_dist: ["datasette>=1.0a20", "httpx"]
requires_python: >=3.10
version: 0.1a3
description:
  # datasette-turnstile

  [![PyPI](https://img.shields.io/pypi/v/datasette-turnstile.svg)](https://pypi.org/project/datasette-turnstile/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-turnstile?include_prereleases&label=changelog)](https://github.com/simonw/datasette-turnstile/releases)
  [![Tests](https://github.com/simonw/datasette-turnstile/actions/workflows/test.yml/badge.svg)](https://github.com/simonw/datasette-turnstile/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-turnstile/blob/main/LICENSE)

  Protect Datasette paths with Cloudflare Turnstile challenges.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-turnstile
  ```

  ## Configuration

  Configure the plugin in your `datasette.yaml`:

  ```yaml
  plugins:
    datasette-turnstile:
      site_key: "0x4AAAAAAxxxxxxxxxxxxxxx"
      secret_key:
        $env: TURNSTILE_SECRET_KEY
      protected_paths:
        - "/admin/*"
        - "/-/import-*"
      exclude_patterns:
        - "*.json"
      cookie_max_age: 86400
  ```

  ### Configuration options

  - **`site_key`** (required): Your Turnstile site key from the Cloudflare dashboard
  - **`secret_key`** (required): Your Turnstile secret key (supports `$env` syntax)
  - **`protected_paths`** (required): List of URL patterns to protect
  - **`exclude_patterns`** (optional): Patterns to exclude from protection (e.g., `*.json`)
  - **`cookie_max_age`** (optional): Cookie lifetime in seconds (default: 86400 = 24 hours)
  - **`cookie_name`** (optional): Name of the verification cookie (default: `ds_turnstile`)

  ### URL Pattern Matching

  Patterns use simple wildcard matching where `*` matches any characters:

  - `/admin/*` - Protects all paths under `/admin/`
  - `/-/import-*` - Protects `/-/import-csv`, `/-/import-json`, etc.
  - `/data?*&*&*` - Protects `/data` with 2+ query string parameters

  Use `?` in patterns to match against the full URL including query string. Without `?`, patterns only match the path.

  ## How It Works

  1. When a user visits a protected path, they're redirected to `/-/turnstile`
  2. The challenge page displays a Cloudflare Turnstile widget
  3. Upon completing the challenge, the token is verified server-side
  4. On success, a signed cookie is set and the user is redirected to their original destination
  5. The cookie remains valid for 24 hours (configurable)

  ### API Requests

  For requests with `Accept: application/json` header, the plugin returns a 403 JSON response instead of redirecting:

  ```json
  {"error": "turnstile_required"}
  ```

  Use `exclude_patterns: ["*.json"]` to exclude JSON endpoints from protection entirely.

  ## Development

  To set up this plugin locally, first checkout the code:
  ```bash
  cd datasette-turnstile
  ```

  To run the tests:
  ```bash
  uv run pytest
  ```

  Create a config file using [Turnstile test keys](https://developers.cloudflare.com/turnstile/troubleshooting/testing/):
  ```bash
  cat > datasette.yaml << 'EOF'
  plugins:
    datasette-turnstile:
      site_key: "1x00000000000000000000AA"
      secret_key:
        $env: TURNSTILE_SECRET_KEY
      protected_paths:
        - "/demo/example*
  EOF
  ```
  Create an example database:
  ```bash
  sqlite3 demo.db "CREATE TABLE example (id INTEGER PRIMARY KEY, name TEXT);"
  ```

  Put the secret in an environment variable and run Datasette with the plugin:
  ```bash
  TURNSTILE_SECRET_KEY='1x0000000000000000000000000000000AA' uv run datasette -c datasette.yaml demo.db
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-turnstile/actions", "Changelog": "https://github.com/simonw/datasette-turnstile/releases", "Homepage": "https://github.com/simonw/datasette-turnstile", "Issues": "https://github.com/simonw/datasette-turnstile/issues"}


name: datasette-unsafe-actor-debug
summary: Debug plugin that lets you imitate any actor
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-unsafe-actor-debug/
project_url: https://pypi.org/project/datasette-unsafe-actor-debug/
release_url: https://pypi.org/project/datasette-unsafe-actor-debug/0.2/
requires_python: >=3.8
version: 0.2
description:
  # datasette-unsafe-actor-debug

  [![PyPI](https://img.shields.io/pypi/v/datasette-unsafe-actor-debug.svg)](https://pypi.org/project/datasette-unsafe-actor-debug/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-unsafe-actor-debug?include_prereleases&label=changelog)](https://github.com/datasette/datasette-unsafe-actor-debug/releases)
  [![Tests](https://github.com/datasette/datasette-unsafe-actor-debug/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-unsafe-actor-debug/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-unsafe-actor-debug/blob/main/LICENSE)

  Debug plugin that lets you imitate any actor. **Only use this for debugging purposes.**

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-unsafe-actor-debug
  ```
  ## Configuration

  You need to set the following plugin configuration to enable this tool:

  ```yaml
  plugins:
    datasette-unsafe-actor-debug:
      enabled: true
  ```

  With Datasette 1.0 alpha you can enable it like this:
  ```bash
  datasette -s plugins.datasette-unsafe-actor-debug.enabled 1
  ```

  ## Usage

  This plugin adds a public page at `/-/unsafe-actor` that lets you sign into Datasette as _any_ actor.

  This option will also be available through the Datasette global navigation menu.

  **Do not enable this tool in production!**

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-unsafe-actor-debug
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-unsafe-actor-debug/actions", "Changelog": "https://github.com/datasette/datasette-unsafe-actor-debug/releases", "Homepage": "https://github.com/datasette/datasette-unsafe-actor-debug", "Issues": "https://github.com/datasette/datasette-unsafe-actor-debug/issues"}

requires_dist:
[cell repeats an earlier one in this file: 113 characters]


name: datasette-unsafe-expose-env
summary: Datasette plugin to expose some environment variables at /-/env for debugging
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-unsafe-expose-env
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-unsafe-expose-env/
project_url: https://pypi.org/project/datasette-unsafe-expose-env/
release_url: https://pypi.org/project/datasette-unsafe-expose-env/0.1.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1.1
description:
  # datasette-unsafe-expose-env

  [![PyPI](https://img.shields.io/pypi/v/datasette-unsafe-expose-env.svg)](https://pypi.org/project/datasette-unsafe-expose-env/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-unsafe-expose-env?include_prereleases&label=changelog)](https://github.com/simonw/datasette-unsafe-expose-env/releases)
  [![Tests](https://github.com/simonw/datasette-unsafe-expose-env/workflows/Test/badge.svg)](https://github.com/simonw/datasette-unsafe-expose-env/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-unsafe-expose-env/blob/main/LICENSE)

  Datasette plugin to expose some environment variables at `/-/env` - for debugging

  > :warning: **This plugin is no longer maintained**. [datasette-expose-env](https://datasette.io/plugins/datasette-expose-env) is recommended as a safe replacement that only exposes environment variables that you have explicitly configured.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-unsafe-expose-env

  ## Usage

  Once installed, the plugin will show a list of environment variables at `/-/env`

  It will redact the following variables by default:

  - `DATABASE_URL`
  - `GPG_KEY`
  - `DATASETTE_SECRET`

  You can configure an alternative set of redacted secrets in your `metadata.yml` configuration:

  ```yaml
  plugins:
    datasette-unsafe-expose-env:
      redact:
      - DATASETTE_SECRET
      - GPG_KEY
      - MY_OTHER_SECRET
  ```

  Once again though: only use this plugin if you need it for debugging purposes and are absolutely certain it won't expose any valuable information. If in doubt, do not use this at all.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-unsafe-expose-env
      python3 -mvenv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-unsafe-expose-env/actions", "Changelog": "https://github.com/simonw/datasette-unsafe-expose-env/releases", "Homepage": "https://github.com/simonw/datasette-unsafe-expose-env", "Issues": "https://github.com/simonw/datasette-unsafe-expose-env/issues"}


name: datasette-upgrade
summary: Upgrade Datasette instance configuration to handle new features
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-upgrade/
project_url: https://pypi.org/project/datasette-upgrade/
release_url: https://pypi.org/project/datasette-upgrade/0.1a0/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.8
version: 0.1a0
description:
  # datasette-upgrade

  [![PyPI](https://img.shields.io/pypi/v/datasette-upgrade.svg)](https://pypi.org/project/datasette-upgrade/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-upgrade?include_prereleases&label=changelog)](https://github.com/datasette/datasette-upgrade/releases)
  [![Tests](https://github.com/datasette/datasette-upgrade/workflows/Test/badge.svg)](https://github.com/datasette/datasette-upgrade/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-upgrade/blob/main/LICENSE)

  Upgrade Datasette instance configuration to handle new features

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-upgrade
  ```
  ## Usage

  The plugin currently provides one command, which will split an old style `metadata.json` or `metadata.yml` file - which contains both metadata and plugin configuration and permissions, into a Datasette 1.0+ `metadata.yml` file and a `datasette.yml` file.

  ```bash
  datasette upgrade metadata-to-config metadata.json -m metadata.yml -c datasette.yml
  ```
  This will leave `metadata.json` in place, but will write out `metadata.yml` and `datasette.yml` as two new files.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-upgrade
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-upgrade/actions", "Changelog": "https://github.com/datasette/datasette-upgrade/releases", "Homepage": "https://github.com/datasette/datasette-upgrade", "Issues": "https://github.com/datasette/datasette-upgrade/issues"}


name: datasette-upload-csvs
summary: Datasette plugin for uploading CSV files and converting them to database tables
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/plugins/datasette-upload-csvs
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-upload-csvs/
project_url: https://pypi.org/project/datasette-upload-csvs/
release_url: https://pypi.org/project/datasette-upload-csvs/0.9.1/
requires_python: >=3.7
version: 0.9.1
description:
  # datasette-upload-csvs

  [![PyPI](https://img.shields.io/pypi/v/datasette-upload-csvs.svg)](https://pypi.org/project/datasette-upload-csvs/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-upload-csvs?include_prereleases&label=changelog)](https://github.com/simonw/datasette-upload-csvs/releases)
  [![Tests](https://github.com/simonw/datasette-upload-csvs/workflows/Test/badge.svg)](https://github.com/simonw/datasette-upload-csvs/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-upload-csvs/blob/main/LICENSE)

  Datasette plugin for uploading CSV files and converting them to database tables

  ## Installation

      datasette install datasette-upload-csvs

  ## Usage

  The plugin adds an interface at `/-/upload-csvs` for uploading a CSV file and using it to create a new database table.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `upload-csvs` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

project_urls:
  {"CI": "https://github.com/simonw/datasette-upload-csvs/actions", "Changelog": "https://github.com/simonw/datasette-upload-csvs/releases", "Homepage": "https://datasette.io/plugins/datasette-upload-csvs", "Issues": "https://github.com/simonw/datasette-upload-csvs/issues"}

requires_dist:
  ["datasette >=0.61", "asgi-csrf >=0.7", "starlette", "aiofiles", "python-multipart", "charset-normalizer", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgiref ; extra == 'test'", "httpx ; extra == 'test'", "asgi-lifespan ; extra == 'test'"]


name: datasette-upload-dbs
summary: Upload SQLite database files to Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-upload-dbs
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-upload-dbs/
project_url: https://pypi.org/project/datasette-upload-dbs/
release_url: https://pypi.org/project/datasette-upload-dbs/0.3.2/
requires_dist: ["datasette", "starlette", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]
requires_python: >=3.7
version: 0.3.2
description:
  # datasette-upload-dbs

  [![PyPI](https://img.shields.io/pypi/v/datasette-upload-dbs.svg)](https://pypi.org/project/datasette-upload-dbs/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-upload-dbs?include_prereleases&label=changelog)](https://github.com/simonw/datasette-upload-dbs/releases)
  [![Tests](https://github.com/simonw/datasette-upload-dbs/workflows/Test/badge.svg)](https://github.com/simonw/datasette-upload-dbs/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-upload-dbs/blob/main/LICENSE)

  Upload SQLite database files to Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-upload-dbs

  ## Configuration

  This plugin requires you to configure a directory in which uploaded files will be stored.

  On startup, Datasette will automatically load any SQLite files that it finds in that directory. This means it is safe to restart your server in between file uploads.

  To configure the directory as `/home/datasette/uploads`, add this to a `metadata.yml` configuration file:

  ```yaml
  plugins:
    datasette-upload-dbs:
      directory: /home/datasette/uploads
  ```

  Or if you are using `metadata.json`:

  ```json
  {
    "plugins": {
      "datasette-upload-dbs": {
        "directory": "/home/datasette/uploads"
      }
    }
  }
  ```
  You can use `"."` for the current folder when the server starts, or `"uploads"` for a folder relative to that folder. The folder will be created on startup if it does not already exist.

  Then start Datasette like this:

      datasette -m metadata.yml

  The plugin defaults to loading all databases in the configured directory.

  You can disable this by adding the following setting:
  ```
  "skip_startup_scan": true
  ```
  ## Usage

  Only users with the `upload-dbs` permission will be able to upload files. The `root` user has this permission by default - other users can be granted access using permission plugins, see the [Permissions](https://docs.datasette.io/en/stable/authentication.html#permissions) documentation for details.

  To start Datasette as the root user, run this:

      datasette -m metadata.yml --root

  And follow the link that is displayd on the console.

  If a user has that permission they will see an "Upload database" link in the navigation menu.

  This will take them to `/-/upload-dbs` where they will be able to upload database files, by selecting them or by dragging them onto the drop area.

  ![Animated demo showing a file being dropped onto a box, then uploading and redirecting to the database page](https://github.com/simonw/datasette-upload-dbs/raw/main/upload-demo.gif)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-upload-dbs
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-upload-dbs/actions", "Changelog": "https://github.com/simonw/datasette-upload-dbs/releases", "Homepage": "https://github.com/simonw/datasette-upload-dbs", "Issues": "https://github.com/simonw/datasette-upload-dbs/issues"}


name: datasette-uptime
summary: Datasette plugin showing uptime at /-/uptime
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-uptime/
project_url: https://pypi.org/project/datasette-uptime/
release_url: https://pypi.org/project/datasette-uptime/0.1.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.8
version: 0.1.1
description:
  # datasette-uptime

  [![PyPI](https://img.shields.io/pypi/v/datasette-uptime.svg)](https://pypi.org/project/datasette-uptime/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-uptime?include_prereleases&label=changelog)](https://github.com/datasette/datasette-uptime/releases)
  [![Tests](https://github.com/datasette/datasette-uptime/actions/workflows/test.yml/badge.svg)](https://github.com/datasette/datasette-uptime/actions/workflows/test.yml)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-uptime/blob/main/LICENSE)

  Datasette plugin showing uptime at /-/uptime

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-uptime
  ```
  ## Usage

  Adds an endpint at `/-/uptime` returning JSON that looks like this:
  ```json
  {
      "started": 355764.709263208,
      "now": 355824.192608208,
      "uptime_seconds": 59.483345208049286,
      "uptime_hours": 0.0165231515625136
  }
  ```
  This shows how long the server has been running.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-uptime
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-uptime/actions", "Changelog": "https://github.com/datasette/datasette-uptime/releases", "Homepage": "https://github.com/datasette/datasette-uptime", "Issues": "https://github.com/datasette/datasette-uptime/issues"}


name: datasette-vega
summary: A Datasette plugin that provides tools for generating charts using Vega
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-vega
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-vega/
project_url: https://pypi.org/project/datasette-vega/
project_urls: {"Homepage": "https://github.com/simonw/datasette-vega"}
release_url: https://pypi.org/project/datasette-vega/0.6.2/
requires_dist: ["datasette"]
version: 0.6.2
description:
  # datasette-vega

  [![PyPI](https://img.shields.io/pypi/v/datasette-vega.svg)](https://pypi.org/project/datasette-vega/)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-vega/blob/master/LICENSE)

  A [Datasette](https://github.com/simonw/datasette) plugin that provides tools
  for generating charts using [Vega](https://vega.github.io/).

  ![Datasette Vega interface](https://raw.githubusercontent.com/simonw/datasette-vega/master/datasette-vega.png)

  Try out the latest master build as a live demo at https://datasette-vega-latest.datasette.io/ or try the latest release installed as a plugin at https://fivethirtyeight.datasettes.com/

  To add this to your Datasette installation, install the plugin like so:

      pip install datasette-vega

  The plugin will then add itself to every Datasette table view.

  If you are publishing data using the `datasette publish` command, you can
  include this plugin like so:

      datasette publish now mydatabase.db --install=datasette-vega

  If you don't want to install any software at all, you can publish your CSV files with Datasette and activate this plugin using the web application at https://publish.datasettes.com/


name: datasette-verify
summary: Verify that SQLite files can be opened using Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-verify
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-verify/
project_url: https://pypi.org/project/datasette-verify/
release_url: https://pypi.org/project/datasette-verify/0.1/
requires_dist: ["datasette (>=0.59a2)", "pytest ; extra == 'test'"]
requires_python: >=3.6
version: 0.1
description:
  # datasette-verify

  [![PyPI](https://img.shields.io/pypi/v/datasette-verify.svg)](https://pypi.org/project/datasette-verify/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-verify?include_prereleases&label=changelog)](https://github.com/simonw/datasette-verify/releases)
  [![Tests](https://github.com/simonw/datasette-verify/workflows/Test/badge.svg)](https://github.com/simonw/datasette-verify/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-verify/blob/main/LICENSE)

  Verify that SQLite files can be opened using Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-verify

  This plugin depends on [Datasette 0.59a2](https://github.com/simonw/datasette/releases/tag/0.59a2) or higher, as it uses the [register_commands()](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-register-commands) plugin hook.

  ## Usage

  To confirm that files can be opened by Datasette, run the following:

      datasette verify file1.db file2.db

  You can pass one or more file paths.

  The command will exit silently with a 0 exit code if the files are all valid SQLite databases that Datasette can open.

  It will exit with a 1 exit code and display an error for the first file it finds that is not valid.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-verify
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-verify/actions", "Changelog": "https://github.com/simonw/datasette-verify/releases", "Homepage": "https://github.com/simonw/datasette-verify", "Issues": "https://github.com/simonw/datasette-verify/issues"}


name: datasette-visible-internal-db
summary: Datasette plugin for making the internal database visible for debugging
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license: Apache-2.0
package_url: https://pypi.org/project/datasette-visible-internal-db/
project_url: https://pypi.org/project/datasette-visible-internal-db/
release_url: https://pypi.org/project/datasette-visible-internal-db/0.1.1/
requires_dist: ["datasette >=1.0a5", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
version: 0.1.1
description:
  # datasette-visible-internal-db

  [![PyPI](https://img.shields.io/pypi/v/datasette-visible-internal-db.svg)](https://pypi.org/project/datasette-visible-internal-db/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-visible-internal-db?include_prereleases&label=changelog)](https://github.com/datasette/datasette-visible-internal-db/releases)
  [![Tests](https://github.com/datasette/datasette-visible-internal-db/workflows/Test/badge.svg)](https://github.com/datasette/datasette-visible-internal-db/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-visible-internal-db/blob/main/LICENSE)

  Datasette plugin for making the internal database visible for debugging

  Datasette 1.0a5 made [the internal database](https://docs.datasette.io/en/1.0a5/internals.html#datasette-s-internal-database) no longer visible through the Datasette interface.

  This plugin brings it back, for debugging purposes.

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  datasette install datasette-visible-internal-db
  ```

  The internal database will now be publicly visible at `/_internal` in your Datasette instance.

  ## Controlling access to the internal database

  You can use Datasette's [permissions mechanism](https://docs.datasette.io/en/stable/authentication.html#permissions) to control who is allowed to interact with the internal database.

  To restrict access to just the `root` actor, drop this into your configuration:

  ```yaml
  databases:
    _internal:
      allow:
        id: root
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasette-visible-internal-db/actions", "Changelog": "https://github.com/datasette/datasette-visible-internal-db/releases", "Homepage": "https://github.com/datasette/datasette-visible-internal-db", "Issues": "https://github.com/datasette/datasette-visible-internal-db/issues"}


name: datasette-write-ui
summary: Datasette UI plugin for editing and inserting rows
classifiers: ["Framework :: Datasette"]
author: Alex Garcia
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette-write-ui/
project_url: https://pypi.org/project/datasette-write-ui/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-write-ui/0.0.1a14/
requires_python: >=3.10
version: 0.0.1a14
description:
  # datasette-write-ui

  [![PyPI](https://img.shields.io/pypi/v/datasette-write-ui.svg)](https://pypi.org/project/datasette-write-ui/)
  [![Changelog](https://img.shields.io/github/v/release/datasette/datasette-write-ui?include_prereleases&label=changelog)](https://github.com/datasette/datasette-write-ui/releases)
  [![Tests](https://github.com/datasette/datasette-write-ui/workflows/Test/badge.svg)](https://github.com/datasette/datasette-write-ui/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/datasette/datasette-write-ui/blob/main/LICENSE)

  A Datasette plugin that adds UI elements to edit, insert, or delete rows in SQLite tables.

  ## Installation

  This plugin requires the alpha version of Datasette 1.0. You can install it with:

  ```bash
  pip install datasette==1.0a3
  ```

  After that, you can install this plugin in the same environment as Datasette.

  ```bash
  datasette install datasette-write-ui
  ```

  ## Usage

  Once installed, the new UI elements will appear on the table page. You'll need special permission in order to see them. The easiest way is to [use the root user](https://docs.datasette.io/en/latest/authentication.html#using-the-root-actor), or otherwise use a [custom actor](https://docs.datasette.io/en/latest/authentication.html#actors).

  <img width=600 src="https://camo.githubusercontent.com/f02dbf756d3ba7ed148a9e7263eb045fafcbb549733f37cd1350aafe54ff2c9f/68747470733a2f2f6461746173657474652d636c6f75642d6173736574732e73332e616d617a6f6e6177732e636f6d2f626c6f672f323032332f696e74726f647563696e672d6461746173657474652d77726974652d75692f75692d7461626c652d73637265656e73686f742e6a7067"/>

  If you actor has the `insert-row` permissions, you'll see the "Insert new row" button at the bottom of the page.

  <video src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/introducing-datasette-write-ui/insert-demo.mp4" width=600 controls autoplay type="video/mp4" ></video>

  If your actor has the `update-row` permissions, you'll see a new "Edit row" option in a new menu on each row.

  <video src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/introducing-datasette-write-ui/edit-demo.mp4" width=600 controls autoplay type="video/mp4" ></video>

  If your actor has the `delete-row` permissions, you'll see a new "Delete row" option in a new menu on each row.

  <video src="https://datasette-cloud-assets.s3.amazonaws.com/blog/2023/introducing-datasette-write-ui/delete-demo.mp4" width=600 controls autoplay type="video/mp4" ></video>

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-write-ui
  python3 -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```
  pip install -e '.[test]'
  ```
  To install the JavaScript build dependencies, run this:
  ```bash
  npm install
  ```
  You can use the [Just](https://github.com/casey/just) command runner to build the TypeScript to minified JavaScript like this:
  ```bash
  just js
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/datasette/datasettewrite-ui/actions", "Changelog": "https://github.com/datasette/datasettewrite-ui/releases", "Homepage": "https://github.com/datasette/datasettewrite-ui", "Issues": "https://github.com/datasette/datasettewrite-ui/issues"}

requires_dist:
  ["datasette>=1.0a21", "sqlite-utils>=3.38", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\""]


name: datasette-write
summary: Datasette plugin providing a UI for writing to a database
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-write
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-write/
project_url: https://pypi.org/project/datasette-write/
provides_extra: ["test"]
release_url: https://pypi.org/project/datasette-write/0.4/
version: 0.4
description:
  # datasette-write

  [![PyPI](https://img.shields.io/pypi/v/datasette-write.svg)](https://pypi.org/project/datasette-write/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-write?label=changelog)](https://github.com/simonw/datasette-write/releases)
  [![Tests](https://github.com/simonw/datasette-write/workflows/Test/badge.svg)](https://github.com/simonw/datasette-write/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-write/blob/master/LICENSE)

  Datasette plugin providing a UI for writing to a database

  ## Installation

  Install this plugin in the same environment as Datasette.
  ```bash
  pip install datasette-write
  ```
  ## Usage

  Having installed the plugin, visit `/db/-/write` on your Datasette instance to submit SQL queries that will be executed against a write connection to the specified database.

  By default only the `root` user can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `datasette-write` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  Pass `?sql=...` in the query string to pre-populate the SQL editor with a query.

  ## Parameterized queries

  SQL queries can include parameters like this:
  ```sql
  insert into news (title, body)
      values (:title, :body_textarea)
  ```
  These will be converted into form fields on the `/db/-/write` page.

  If a parameter name ends with `_textarea` it will be rendered as a multi-line textarea instead of a text input.

  If a parameter name ends with `_hidden` it will be rendered as a hidden input.

  ## Updating rows with SQL

  On Datasette 1.0a13 and higher a row actions menu item will be added to the row page linking to a SQL query for updating that row, for users with the `datasette-write` permission.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:
  ```bash
  cd datasette-write
  python3 -mvenv venv
  source venv/bin/activate
  ```
  Or if you are using `pipenv`:
  ```bash
  pipenv shell
  ```
  Now install the dependencies and tests:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-write/actions", "Changelog": "https://github.com/simonw/datasette-write/releases", "Homepage": "https://github.com/simonw/datasette-write", "Issues": "https://github.com/simonw/datasette-write/issues"}

requires_dist:
  ["datasette>=0.64.6", "pytest; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "httpx; extra == \"test\"", "beautifulsoup4; extra == \"test\""]


name: datasette-x-forwarded-host
summary: Treat the X-Forwarded-Host header as the Host header
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-x-forwarded-host
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-x-forwarded-host/
project_url: https://pypi.org/project/datasette-x-forwarded-host/
release_url: https://pypi.org/project/datasette-x-forwarded-host/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1
description:
  # datasette-x-forwarded-host

  [![PyPI](https://img.shields.io/pypi/v/datasette-x-forwarded-host.svg)](https://pypi.org/project/datasette-x-forwarded-host/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-x-forwarded-host?include_prereleases&label=changelog)](https://github.com/simonw/datasette-x-forwarded-host/releases)
  [![Tests](https://github.com/simonw/datasette-x-forwarded-host/workflows/Test/badge.svg)](https://github.com/simonw/datasette-x-forwarded-host/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-x-forwarded-host/blob/main/LICENSE)

  Treat the X-Forwarded-Host header as the Host header

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-x-forwarded-host

  ## Usage

  Once installed, Datasette will replace the `host` header with the content of the incoming `x-forwarded-host` header.

  This helps Datasette generate links to new pages that work when hosted behind a proxy that rewrites the `host` header.

  Only use this plugin in deployment environmens where you know the `x-forwarded-host` header can be trusted!

  This has been tested on GitHub Codespaces.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-x-forwarded-host
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-x-forwarded-host/actions", "Changelog": "https://github.com/simonw/datasette-x-forwarded-host/releases", "Homepage": "https://github.com/simonw/datasette-x-forwarded-host", "Issues": "https://github.com/simonw/datasette-x-forwarded-host/issues"}


name: datasette-yaml
summary: Export Datasette records as YAML
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-yaml
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-yaml/
project_url: https://pypi.org/project/datasette-yaml/
release_url: https://pypi.org/project/datasette-yaml/0.1.1/
version: 0.1.1
description:
  # datasette-yaml

  [![PyPI](https://img.shields.io/pypi/v/datasette-yaml.svg)](https://pypi.org/project/datasette-yaml/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-yaml?include_prereleases&label=changelog)](https://github.com/simonw/datasette-yaml/releases)
  [![Tests](https://github.com/simonw/datasette-yaml/workflows/Test/badge.svg)](https://github.com/simonw/datasette-yaml/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-yaml/blob/main/LICENSE)

  Export Datasette records as YAML

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-yaml

  ## Usage

  Having installed this plugin, every table and query will gain a new `.yaml` export link.

  You can also construct these URLs directly: `/dbname/tablename.yaml`

  ## Demo

  The plugin is running on [covid-19.datasettes.com](https://covid-19.datasettes.co/) - for example [/covid/latest_ny_times_counties_with_populations.yaml](https://covid-19.datasettes.com/covid/latest_ny_times_counties_with_populations.yaml)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-yaml
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-yaml/actions", "Changelog": "https://github.com/simonw/datasette-yaml/releases", "Homepage": "https://github.com/simonw/datasette-yaml", "Issues": "https://github.com/simonw/datasette-yaml/issues"}

requires_dist:
[cell repeats an earlier one in this file: 138 characters]


name: datasette-youtube-embed
summary: Turn YouTube URLs into embedded players in Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
license_expression: Apache-2.0
package_url: https://pypi.org/project/datasette-youtube-embed/
project_url: https://pypi.org/project/datasette-youtube-embed/
release_url: https://pypi.org/project/datasette-youtube-embed/0.2/
requires_dist: ["datasette"]
requires_python: >=3.10
version: 0.2
description:
  # datasette-youtube-embed

  [![PyPI](https://img.shields.io/pypi/v/datasette-youtube-embed.svg)](https://pypi.org/project/datasette-youtube-embed/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-youtube-embed?include_prereleases&label=changelog)](https://github.com/simonw/datasette-youtube-embed/releases)
  [![Tests](https://github.com/simonw/datasette-youtube-embed/workflows/Test/badge.svg)](https://github.com/simonw/datasette-youtube-embed/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-youtube-embed/blob/main/LICENSE)

  Turn YouTube URLs into embedded players in Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-youtube-embed

  ## Usage

  Once installed, any columns containing URLs that look like one of these:

  - `https://www.youtube.com/watch?v=xyz`
  - `https://www.youtube.com/watch?v=xyz&start=30`
  - `https://www.youtube.com/watch?v=xyz&start=30&end=50`

  Will be turned into YouTube embeds like this:

  ```html
  <lite-youtube videoid="xyz" params="start=30" style="min-width: 200px"></lite-youtube>
  ```
  These will then be rendered using [Lite YouTube Embed](https://github.com/paulirish/lite-youtube-embed), which avoids loading the full YouTube embed until someone interacts with it.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-youtube-embed
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-youtube-embed/actions", "Changelog": "https://github.com/simonw/datasette-youtube-embed/releases", "Homepage": "https://github.com/simonw/datasette-youtube-embed", "Issues": "https://github.com/simonw/datasette-youtube-embed/issues"}


name: datasette
summary: An open source multi-tool for exploring and publishing data
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/
license: Apache License, Version 2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/datasette/
project_url: https://pypi.org/project/datasette/
provides_extra: ["docs", "test", "rich"]
release_url: https://pypi.org/project/datasette/0.65.2/
requires_python: >=3.9
version: 0.65.2
classifiers:
  ["Development Status :: 4 - Beta", "Framework :: Datasette", "Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "Intended Audience :: Science/Research", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.9", "Topic :: Database"]

description:
  <img src="https://datasette.io/static/datasette-logo.svg" alt="Datasette">

  [![PyPI](https://img.shields.io/pypi/v/datasette.svg)](https://pypi.org/project/datasette/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette?label=changelog)](https://docs.datasette.io/en/stable/changelog.html)
  [![Python 3.x](https://img.shields.io/pypi/pyversions/datasette.svg?logo=python&logoColor=white)](https://pypi.org/project/datasette/)
  [![Tests](https://github.com/simonw/datasette/workflows/Test/badge.svg)](https://github.com/simonw/datasette/actions?query=workflow%3ATest)
  [![Documentation Status](https://readthedocs.org/projects/datasette/badge/?version=latest)](https://docs.datasette.io/en/latest/?badge=latest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette/blob/main/LICENSE)
  [![docker: datasette](https://img.shields.io/badge/docker-datasette-blue)](https://hub.docker.com/r/datasetteproject/datasette)
  [![discord](https://img.shields.io/discord/823971286308356157?label=discord)](https://discord.gg/ktd74dm5mw)

  *An open source multi-tool for exploring and publishing data*

  Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size and publish that as an interactive, explorable website and accompanying API.

  Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with the world.

  [Explore a demo](https://global-power-plants.datasettes.com/global-power-plants/global-power-plants), watch [a video about the project](https://simonwillison.net/2021/Feb/7/video/) or try it out by [uploading and publishing your own CSV data](https://docs.datasette.io/en/stable/getting_started.html#try-datasette-without-installing-anything-using-glitch).

  * [datasette.io](https://datasette.io/) is the official project website
  * Latest [Datasette News](https://datasette.io/news)
  * Comprehensive documentation: https://docs.datasette.io/
  * Examples: https://datasette.io/examples
  * Live demo of current `main` branch: https://latest.datasette.io/
  * Questions, feedback or want to talk about the project? Join our [Discord](https://discord.gg/ktd74dm5mw)

  Want to stay up-to-date with the project? Subscribe to the [Datasette newsletter](https://datasette.substack.com/) for tips, tricks and news on what's new in the Datasette ecosystem.

  ## Installation

  If you are on a Mac, [Homebrew](https://brew.sh/) is the easiest way to install Datasette:

      brew install datasette

  You can also install it using `pip` or `pipx`:

      pip install datasette

  Datasette requires Python 3.7 or higher. We also have [detailed installation instructions](https://docs.datasette.io/en/stable/installation.html) covering other options such as Docker.

  ## Basic usage

      datasette serve path/to/database.db

  This will start a web server on port 8001 - visit http://localhost:8001/ to access the web interface.

  `serve` is the default subcommand, you can omit it if you like.

  Use Chrome on OS X? You can run datasette against your browser history like so:

       datasette ~/Library/Application\ Support/Google/Chrome/Default/History --nolock

  Now visiting http://localhost:8001/History/downloads will show you a web interface to browse your downloads data:

  ![Downloads table rendered by datasette](https://static.simonwillison.net/static/2017/datasette-downloads.png)

  ## metadata.json

  If you want to include licensing and source information in the generated datasette website you can do so using a JSON file that looks something like this:

      {
          "title": "Five Thirty Eight",
          "license": "CC Attribution 4.0 License",
          "license_url": "http://creativecommons.org/licenses/by/4.0/",
          "source": "fivethirtyeight/data on GitHub",
          "source_url": "https://github.com/fivethirtyeight/data"
      }

  Save this in `metadata.json` and run Datasette like so:

      datasette serve fivethirtyeight.db -m metadata.json

  The license and source information will be displayed on the index page and in the footer. They will also be included in the JSON produced by the API.

  ## datasette publish

  If you have [Heroku](https://heroku.com/) or [Google Cloud Run](https://cloud.google.com/run/) configured, Datasette can deploy one or more SQLite databases to the internet with a single command:

      datasette publish heroku database.db

  Or:

      datasette publish cloudrun database.db

  This will create a docker image containing both the datasette application and the specified SQLite database files. It will then deploy that image to Heroku or Cloud Run and give you a URL to access the resulting website and API.

  See [Publishing data](https://docs.datasette.io/en/stable/publish.html) in the documentation for more details.

  ## Datasette Lite

  [Datasette Lite](https://lite.datasette.io/) is Datasette packaged using WebAssembly so that it runs entirely in your browser, no Python web application server required. Read more about that in the [Datasette Lite documentation](https://github.com/simonw/datasette-lite/blob/main/README.md).

dynamic:
  ["Author", "Classifier", "Description", "Description-Content-Type", "Home-Page", "License", "License-File", "Project-Url", "Provides-Extra", "Requires-Dist", "Requires-Python", "Summary"]

project_urls:
  {"CI": "https://github.com/simonw/datasette/actions?query=workflow%3ATest", "Changelog": "https://docs.datasette.io/en/stable/changelog.html", "Documentation": "https://docs.datasette.io/en/stable/", "Homepage": "https://datasette.io/", "Issues": "https://github.com/simonw/datasette/issues", "Live demo": "https://latest.datasette.io/", "Source code": "https://github.com/simonw/datasette"}

requires_dist:
  ["asgiref>=3.2.10", "click>=7.1.1", "click-default-group>=1.2.3", "Jinja2>=2.10.3", "hupper>=1.9", "httpx>=0.20", "pluggy>=1.0", "uvicorn>=0.11", "aiofiles>=0.4", "janus>=0.6.2", "asgi-csrf>=0.9", "PyYAML>=5.3", "mergedeep>=1.1.1", "itsdangerous>=1.1", "setuptools", "pip", "platformdirs>=2.1.0", "typing_extensions>=4.0.0", "flexcache>=0.3", "flexparser>=0.3", "furo==2024.8.6; extra == \"docs\"", "sphinx-autobuild; extra == \"docs\"", "codespell; extra == \"docs\"", "blacken-docs; extra == \"docs\"", "sphinx-copybutton; extra == \"docs\"", "pytest>=5.2.2; extra == \"test\"", "pytest-xdist>=2.2.1; extra == \"test\"", "pytest-asyncio>=0.17; extra == \"test\"", "beautifulsoup4>=4.8.1; extra == \"test\"", "black==24.4.2; extra == \"test\"", "blacken-docs==1.12.1; extra == \"test\"", "pytest-timeout>=1.4.2; extra == \"test\"", "trustme>=0.7; extra == \"test\"", "cogapp>=3.3.0; extra == \"test\"", "rich; extra == \"rich\""]


name: db-to-sqlite
summary: CLI tool for exporting tables or queries from any SQL database to a SQLite file
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/tools/db-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/db-to-sqlite/
project_url: https://pypi.org/project/db-to-sqlite/
release_url: https://pypi.org/project/db-to-sqlite/1.5/
requires_python: >=3.7
version: 1.5
classifiers:
  ["Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Database"]

description:
  # db-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/db-to-sqlite.svg)](https://pypi.python.org/pypi/db-to-sqlite)
  [![Changelog](https://img.shields.io/github/v/release/simonw/db-to-sqlite?include_prereleases&label=changelog)](https://github.com/simonw/db-to-sqlite/releases)
  [![Tests](https://github.com/simonw/db-to-sqlite/workflows/Test/badge.svg)](https://github.com/simonw/db-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/db-to-sqlite/blob/main/LICENSE)

  CLI tool for exporting tables or queries from any SQL database to a SQLite file.

  ## Installation

  Install from PyPI like so:

      pip install db-to-sqlite

  If you want to use it with MySQL, you can install the extra dependency like this:

      pip install 'db-to-sqlite[mysql]'

  Installing the `mysqlclient` library on OS X can be tricky - I've found [this recipe](https://gist.github.com/simonw/90ac0afd204cd0d6d9c3135c3888d116) to work (run that before installing `db-to-sqlite`).

  For PostgreSQL, use this:

      pip install 'db-to-sqlite[postgresql]'

  ## Usage
  ```
  Usage: db-to-sqlite [OPTIONS] CONNECTION PATH

    Load data from any database into SQLite.

    PATH is a path to the SQLite file to create, e.c. /tmp/my_database.db

    CONNECTION is a SQLAlchemy connection string, for example:

        postgresql://localhost/my_database
        postgresql://username:passwd@localhost/my_database

        mysql://root@localhost/my_database
        mysql://username:passwd@localhost/my_database

    More: https://docs.sqlalchemy.org/en/13/core/engines.html#database-urls

  Options:
    --version                     Show the version and exit.
    --all                         Detect and copy all tables
    --table TEXT                  Specific tables to copy
    --skip TEXT                   When using --all skip these tables
    --redact TEXT...              (table, column) pairs to redact with ***
    --sql TEXT                    Optional SQL query to run
    --output TEXT                 Table in which to save --sql query results
    --pk TEXT                     Optional column to use as a primary key
    --index-fks / --no-index-fks  Should foreign keys have indexes? Default on
    -p, --progress                Show progress bar
    --postgres-schema TEXT        PostgreSQL schema to use
    --help                        Show this message and exit.
  ```

  For example, to save the content of the `blog_entry` table from a PostgreSQL database to a local file called `blog.db` you could do this:

      db-to-sqlite "postgresql://localhost/myblog" blog.db \
          --table=blog_entry

  You can specify `--table` more than once.

  You can also save the data from all of your tables, effectively creating a SQLite copy of your entire database. Any foreign key relationships will be detected and added to the SQLite database. For example:

      db-to-sqlite "postgresql://localhost/myblog" blog.db \
          --all

  When running `--all` you can specify tables to skip using `--skip`:

      db-to-sqlite "postgresql://localhost/myblog" blog.db \
          --all \
          --skip=django_migrations

  If you want to save the results of a custom SQL query, do this:

      db-to-sqlite "postgresql://localhost/myblog" output.db \
          --output=query_results \
          --sql="select id, title, created from blog_entry" \
          --pk=id

  The `--output` option specifies the table that should contain the results of the query.

  ## Using db-to-sqlite with PostgreSQL schemas

  If the tables you want to copy from your PostgreSQL database aren't in the default schema, you can specify an alternate one with the `--postgres-schema` option:

      db-to-sqlite "postgresql://localhost/myblog" blog.db \
          --all \
          --postgres-schema my_schema

  ## Using db-to-sqlite with MS SQL

  The best way to get the connection string needed for the MS SQL connections below is to use urllib from the Standard Library as below

      params = urllib.parse.quote_plus(
          "DRIVER={SQL Server Native Client 11.0};"
          "SERVER=localhost;"
          "DATABASE=my_database;"
          "Trusted_Connection=yes;"
      )

  The above will resolve to

      DRIVER%3D%7BSQL+Server+Native+Client+11.0%7D%3B+SERVER%3Dlocalhost%3B+DATABASE%3Dmy_database%3B+Trusted_Connection%3Dyes

  You can then use the string above in the odbc_connect below

      mssql+pyodbc:///?odbc_connect=DRIVER%3D%7BSQL+Server+Native+Client+11.0%7D%3B+SERVER%3Dlocalhost%3B+DATABASE%3Dmy_database%3B+Trusted_Connection%3Dyes
      mssql+pyodbc:///?odbc_connect=DRIVER%3D%7BSQL+Server+Native+Client+11.0%7D%3B+SERVER%3Dlocalhost%3B+DATABASE%3Dmy_database%3B+UID%3Dusername%3B+PWD%3Dpasswd

  ## Using db-to-sqlite with Heroku Postgres

  If you run an application on [Heroku](https://www.heroku.com/) using their [Postgres database product](https://www.heroku.com/postgres), you can use the `heroku config` command to access a compatible connection string:

      $ heroku config --app myappname | grep HEROKU_POSTG
      HEROKU_POSTGRESQL_OLIVE_URL: postgres://username:password@ec2-xxx-xxx-xxx-x.compute-1.amazonaws.com:5432/dbname

  You can pass this to `db-to-sqlite` to create a local SQLite database with the data from your Heroku instance.

  You can even do this using a bash one-liner:

      $ db-to-sqlite $(heroku config --app myappname | grep HEROKU_POSTG | cut -d: -f 2-) \
          /tmp/heroku.db --all -p
      1/23: django_migrations
      ...
      17/23: blog_blogmark
      [####################################]  100%
      ...

  ## Related projects

  * [Datasette](https://github.com/simonw/datasette): A tool for exploring and publishing data. Works great with SQLite files generated using `db-to-sqlite`.
  * [sqlite-utils](https://github.com/simonw/sqlite-utils): Python CLI utility and library for manipulating SQLite databases.
  * [csvs-to-sqlite](https://github.com/simonw/csvs-to-sqlite): Convert CSV files into a SQLite database.

  ## Development

  To set up this tool locally, first checkout the code. Then create a new virtual environment:

      cd db-to-sqlite
      python3 -m venv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

  This will skip tests against MySQL or PostgreSQL if you do not have their additional dependencies installed.

  You can install those extra dependencies like so:

      pip install -e '.[test_mysql,test_postgresql]'

  You can alternative use `pip install psycopg2-binary` if you cannot install the `psycopg2` dependency used by the `test_postgresql` extra.

  See [Running a MySQL server using Homebrew](https://til.simonwillison.net/homebrew/mysql-homebrew) for tips on running the tests against MySQL on macOS, including how to install the `mysqlclient` dependency.

  The PostgreSQL and MySQL tests default to expecting to run against servers on localhost. You can use environment variables to point them at different test database servers:

  - `MYSQL_TEST_DB_CONNECTION` - defaults to `mysql://root@localhost/test_db_to_sqlite`
  - `POSTGRESQL_TEST_DB_CONNECTION` - defaults to `postgresql://localhost/test_db_to_sqlite`

  The database you indicate in the environment variable - `test_db_to_sqlite` by default - will be deleted and recreated on every test run.

project_urls:
  {"CI": "https://travis-ci.com/simonw/db-to-sqlite", "Changelog": "https://github.com/simonw/db-to-sqlite/releases", "Documentation": "https://github.com/simonw/db-to-sqlite/blob/main/README.md", "Homepage": "https://datasette.io/tools/db-to-sqlite", "Issues": "https://github.com/simonw/db-to-sqlite/issues", "Source code": "https://github.com/simonw/db-to-sqlite"}

requires_dist:
  ["sqlalchemy", "sqlite-utils (>=2.9.1)", "click", "mysqlclient ; extra == 'mysql'", "psycopg2 ; extra == 'postgresql'", "pytest ; extra == 'test'", "pytest-timeout ; extra == 'test'", "pytest ; extra == 'test_mysql'", "mysqlclient ; extra == 'test_mysql'", "pytest ; extra == 'test_postgresql'", "psycopg2 ; extra == 'test_postgresql'"]


name: dbf-to-sqlite
summary: CLCLI tool for converting DBF files (dBase, FoxPro etc) to SQLite
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/dbf-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/dbf-to-sqlite/
project_url: https://pypi.org/project/dbf-to-sqlite/
project_urls: {"Homepage": "https://github.com/simonw/dbf-to-sqlite"}
release_url: https://pypi.org/project/dbf-to-sqlite/0.1/
requires_dist: ["dbf (==0.97.11)", "click", "sqlite-utils"]
version: 0.1
classifiers:
  ["Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Database"]

description:
  # dbf-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/dbf-to-sqlite.svg)](https://pypi.python.org/pypi/dbf-to-sqlite)
  [![Travis CI](https://travis-ci.com/simonw/dbf-to-sqlite.svg?branch=master)](https://travis-ci.com/simonw/dbf-to-sqlite)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/dbf-to-sqlite/blob/master/LICENSE)


  CLI tool for converting DBF files (dBase, FoxPro etc) to SQLite.

      $ dbf-to-sqlite --help
      Usage: dbf-to-sqlite [OPTIONS] DBF_PATHS... SQLITE_DB

        Convert DBF files (dBase, FoxPro etc) to SQLite

        https://github.com/simonw/dbf-to-sqlite

      Options:
        --version      Show the version and exit.
        --table TEXT   Table name to use (only valid for single files)
        -v, --verbose  Show what's going on
        --help         Show this message and exit.

  Example usage:

      $ dbf-to-sqlite *.DBF database.db

  This will create a new SQLite database called `database.db` containing one table for each of the `DBF` files in the current directory.

  Looking for DBF files to try this out on? Try downloading the [Himalayan Database](http://himalayandatabase.com/) of all expeditions that have climbed in the Nepal Himalaya.


name: dclient
summary: A client CLI utility for Datasette instances
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/dclient
license: Apache License, Version 2.0
package_url: https://pypi.org/project/dclient/
project_url: https://pypi.org/project/dclient/
release_url: https://pypi.org/project/dclient/0.4/
requires_python: >=3.8
version: 0.4
description:
  # dclient

  [![PyPI](https://img.shields.io/pypi/v/dclient.svg)](https://pypi.org/project/dclient/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/dclient?include_prereleases&label=changelog)](https://github.com/simonw/dclient/releases)
  [![Tests](https://github.com/simonw/dclient/workflows/Test/badge.svg)](https://github.com/simonw/dclient/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/dclient/blob/master/LICENSE)

  A client CLI utility for [Datasette](https://datasette.io/) instances.

  Much of the functionality requires Datasette 1.0a2 or higher.

  ## Things you can do with dclient

  - Run SQL queries against Datasette and returning the results as JSON
  - Run queries against authenticated Datasette instances
  - Create aliases and store authentication tokens for convenient access to Datasette
  - Insert data into Datasette using the [insert API](https://docs.datasette.io/en/latest/json_api.html#the-json-write-api) (Datasette 1.0 alpha or higher)

  ## Installation

  Install this tool using `pip`:
  ```bash
  pip install dclient
  ```
  If you want to install it in the same virtual environment as Datasette (to use it as a plugin) you can instead run:
  ```bash
  datasette install dclient
  ```
  ## Running a query

  ```bash
  dclient query https://latest.datasette.io/fixtures "select * from facetable limit 1"
  ```
  To shorten that, create an alias:
  ```bash
  dclient alias add fixtures https://latest.datasette.io/fixtures
  ```
  Then run it like this instead:
  ```bash
  dclient query fixtures "select * from facetable limit 1"
  ```
  ## Documentation

  Visit **[dclient.datasette.io](https://dclient.datasette.io)** for full documentation on using this tool.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:
  ```bash
  cd dclient
  python -m venv venv
  source venv/bin/activate
  ```
  Now install the dependencies and test dependencies:
  ```bash
  pip install -e '.[test]'
  ```
  To run the tests:
  ```bash
  pytest
  ```

project_urls:
  {"CI": "https://github.com/simonw/dclient/actions", "Changelog": "https://github.com/simonw/dclient/releases", "Homepage": "https://github.com/simonw/dclient", "Issues": "https://github.com/simonw/dclient/issues"}

requires_dist:
  ["click", "httpx", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-httpx ; extra == 'test'", "cogapp ; extra == 'test'", "pytest-mock ; extra == 'test'", "datasette >=1.0a2 ; extra == 'test'"]


name: django-sql-dashboard
summary: Django app for building dashboards using raw SQL queries
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/django-sql-dashboard
license: Apache License, Version 2.0
package_url: https://pypi.org/project/django-sql-dashboard/
project_url: https://pypi.org/project/django-sql-dashboard/
release_url: https://pypi.org/project/django-sql-dashboard/1.2/
requires_python: >=3.6
version: 1.2
description:
  # django-sql-dashboard

  [![PyPI](https://img.shields.io/pypi/v/django-sql-dashboard.svg)](https://pypi.org/project/django-sql-dashboard/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/django-sql-dashboard?include_prereleases&label=changelog)](https://github.com/simonw/django-sql-dashboard/releases)
  [![Tests](https://github.com/simonw/django-sql-dashboard/workflows/Test/badge.svg)](https://github.com/simonw/django-sql-dashboard/actions?query=workflow%3ATest)
  [![Documentation Status](https://readthedocs.org/projects/django-sql-dashboard/badge/?version=latest)](http://django-sql-dashboard.datasette.io/en/latest/?badge=latest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/django-sql-dashboard/blob/main/LICENSE)

  Django SQL Dashboard provides an authenticated interface for executing read-only SQL queries directly against your PostgreSQL database, bringing a useful subset of [Datasette](https://datasette.io/) to Django.

  Applications include ad-hoc analysis and debugging, plus the creation of reporting dashboards that can be shared with team members or published online.

  See my blog for [more about this project](https://simonwillison.net/2021/May/10/django-sql-dashboard/), including [a video demo](https://www.youtube.com/watch?v=ausrmMZkPEY).

  Features include:

  - Safely run read-only one or more SQL queries against your database and view the results in your browser
  - Bookmark queries and share those links with other members of your team
  - Create [saved dashboards](https://django-sql-dashboard.datasette.io/en/latest/saved-dashboards.html) from your queries, with full control over who can view and edit them
  - [Named parameters](https://django-sql-dashboard.datasette.io/en/latest/sql.html#sql-parameters) such as `select * from entries where id = %(id)s` will be turned into form fields, allowing quick creation of interactive dashboards
  - Produce [bar charts](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#bar-label-bar-quantity), [progress bars](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#total-count-completed-count) and more from SQL queries, with the ability to easily create new [custom dashboard widgets](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#custom-widgets) using the Django template system
  - Write SQL queries that safely construct and render [markdown](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#markdown) and [HTML](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#html)
  - Export the full results of a SQL query as a downloadable CSV or TSV file, using a combination of Django's [streaming HTTP response](https://docs.djangoproject.com/en/3.2/ref/request-response/#django.http.StreamingHttpResponse) mechanism and PostgreSQL [server-side cursors](https://www.psycopg.org/docs/usage.html#server-side-cursors) to efficiently stream large amounts of data without running out of resources
  - Copy and paste the results of SQL queries directly into tools such as Google Sheets or Excel
  - Uses Django's authentication system, so dashboard accounts can be granted using Django's Admin tools

  ## Documentation

  Full documentation is at [django-sql-dashboard.datasette.io](https://django-sql-dashboard.datasette.io/)

  ## Screenshot

  <img width="1018" alt="Screenshot showing a SQL query that produces a table and one that produces a bar chart" src="https://user-images.githubusercontent.com/9599/124050883-42ad2300-d9d0-11eb-83e6-44ad85f7ef64.png">

  ## Alternatives

  - [django-sql-explorer](https://github.com/groveco/django-sql-explorer) provides a related set of functionality that also works against database backends other than PostgreSQL

project_urls:
  {"CI": "https://github.com/simonw/django-sql-dashboard/actions", "Changelog": "https://github.com/simonw/django-sql-dashboard/releases", "Documentation": "https://django-sql-dashboard.datasette.io/", "Homepage": "https://github.com/simonw/django-sql-dashboard", "Issues": "https://github.com/simonw/django-sql-dashboard/issues"}

requires_dist:
  ["Django >=3.0", "markdown", "bleach", "black >=22.3.0 ; extra == 'test'", "psycopg2 ; extra == 'test'", "pytest ; extra == 'test'", "pytest-django ==4.2.0 ; extra == 'test'", "pytest-pythonpath ; extra == 'test'", "dj-database-url ; extra == 'test'", "testing.postgresql ; extra == 'test'", "beautifulsoup4 ; extra == 'test'", "html5lib ; extra == 'test'"]


name: dogsheep-beta
classifiers: []
author: Simon Willison
description_content_type: text/markdown
license_expression: Apache-2.0
package_url: https://pypi.org/project/dogsheep-beta/
project_url: https://pypi.org/project/dogsheep-beta/
release_url: https://pypi.org/project/dogsheep-beta/0.11/
requires_dist: ["datasette>=0.50.2", "click", "pyyaml", "sqlite-utils>=3.0"]
requires_python: >=3.10
version: 0.11
summary:
  Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette

description:
  # dogsheep-beta

  [![PyPI](https://img.shields.io/pypi/v/dogsheep-beta.svg)](https://pypi.org/project/dogsheep-beta/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/beta?include_prereleases&label=changelog)](https://github.com/dogsheep/beta/releases)
  [![Tests](https://github.com/dogsheep/beta/workflows/Test/badge.svg)](https://github.com/dogsheep/beta/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/beta/blob/main/LICENSE)

  Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette

  ## Example

  A live example of this plugin is running at https://datasette.io/-/beta - configured using [this YAML file](https://github.com/simonw/datasette.io/blob/main/templates/dogsheep-beta.yml).

  Read more about how this example works in [Building a search engine for datasette.io](https://simonwillison.net/2020/Dec/19/dogsheep-beta/).

  ## Installation

  Install this tool like so:

      $ pip install dogsheep-beta

  ## Usage

  Run the indexer using the `dogsheep-beta` command-line tool:

      $ dogsheep-beta index dogsheep.db config.yml

  The `config.yml` file contains details of the databases and document types that should be indexed:

  ```yaml
  twitter.db:
      tweets:
          sql: |-
              select
                  tweets.id as key,
                  'Tweet by @' || users.screen_name as title,
                  tweets.created_at as timestamp,
                  tweets.full_text as search_1
              from tweets join users on tweets.user = users.id
      users:
          sql: |-
              select
                  id as key,
                  name || ' @' || screen_name as title,
                  created_at as timestamp,
                  description as search_1
              from users
  ```

  This will create a `search_index` table in the `dogsheep.db` database populated by data from those SQL queries.

  By default the search index that this tool creates will be configured for Porter stemming. This means that searches for words like `run` will match documents containing `runs` or `running`.

  If you don't want to use Porter stemming, use the `--tokenize none` option:

      $ dogsheep-beta index dogsheep.db config.yml --tokenize none

  You can pass other SQLite tokenize argumenst here, see [the SQLite FTS tokenizers documentation](https://www.sqlite.org/fts5.html#tokenizers).

  ## Columns

  The columns that can be returned by our query are:

  - `key` - a unique (within that type) primary key
  - `title` - the title for the item
  - `timestamp` - an ISO8601 timestamp, e.g. `2020-09-02T21:00:21`
  - `search_1` - a larger chunk of text to be included in the search index
  - `category` - an integer category ID, see below
  - `is_public` - an integer (0 or 1, defaults to 0 if not set) specifying if this is public or not

  Public records are things like your public tweets, blog posts and GitHub commits.

  ## Categories

  Indexed items can be assigned a category. Categories are integers that correspond to records in the `categories` table, which defaults to containing the following:

  |   id | name       |
  |------|------------|
  |    1 | created    |
  |    2 | saved      |
  |    3 | received   |

  `created` is for items that have been created by the Dogsheep instance owner.

  `saved` is for items that they have saved, liked or favourited.

  `received` is for items that have been specifically sent to them by other people - incoming emails or direct messages for example.

  ## Datasette plugin

  Run `datasette install dogsheep-beta` (or use `pip install dogsheep-beta` in the same environment as Datasette) to install the Dogsheep Beta Datasette plugin.

  Once installed, a custom search interface will be made available at `/-/beta`. You can use this interface to execute searches.

  The Datasette plugin has some configuration options. You can set these by adding the following to your `metadata.json` configuration file:

  ```json
  {
      "plugins": {
          "dogsheep-beta": {
              "database": "beta",
              "config_file": "dogsheep-beta.yml",
              "template_debug": true
          }
      }
  }
  ```
  The configuration settings for the plugin are:
  - `database` - the database file that contains your search index. If the file is `beta.db` you should set `database` to `beta`.
  - `config_file` - the YAML file containing your Dogsheep Beta configuration.
  - `template_debug` - set this to `true` to enable debugging output if errors occur in your custom templates, see below.

  ## Custom results display

  Each indexed item type can define custom display HTML as part of the `config.yml` file. It can do this using a `display` key containing a fragment of Jinja template, and optionally a `display_sql` key with extra SQL to execute to fetch the data to display.

  Here's how to define a custom display template for a tweet:

  ```yaml
  twitter.db:
      tweets:
          sql: |-
              select
                  tweets.id as key,
                  'Tweet by @' || users.screen_name as title,
                  tweets.created_at as timestamp,
                  tweets.full_text as search_1
              from tweets join users on tweets.user = users.id
          display: |-
              <p>{{ title }} - tweeted at {{ timestamp }}</p>
              <blockquote>{{ search_1 }}</blockquote>
  ```
  This example reuses the value that were stored in the `search_index` table when the indexing query was run.

  To load in extra values to display in the template, use a `display_sql` query like this:

  ```yaml
  twitter.db:
      tweets:
          sql: |-
              select
                  tweets.id as key,
                  'Tweet by @' || users.screen_name as title,
                  tweets.created_at as timestamp,
                  tweets.full_text as search_1
              from tweets join users on tweets.user = users.id
          display_sql: |-
              select
                  users.screen_name,
                  tweets.full_text,
                  tweets.created_at
              from
                  tweets join users on tweets.user = users.id
              where
                  tweets.id = :key
          display: |-
              <p>{{ display.screen_name }} - tweeted at {{ display.created_at }}</p>
              <blockquote>{{ display.full_text }}</blockquote>
  ```
  The `display_sql` query will be executed for every search result, passing the key value from the `search_index` table as the `:key` parameter and the user's search term as the `:q` parameter.

  This performs well because [many small queries are efficient in SQLite](https://www.sqlite.org/np1queryprob.html).

  If an error occurs while rendering one of your templates the search results page will return a 500 error. You can use the `template_debug` configuration setting described above to instead output debugging information for the search results item that experienced the error.

  ## Displaying maps

  This plugin will eventually include a number of useful shortcuts for rendering interesting content.

  The first available shortcut is for displaying maps. Make your custom content output something like this:

  ```html
  <div
      data-map-latitude="{{ display.latitude }}"
      data-map-longitude="{{ display.longitude }}"
      style="display: none; float: right; width: 250px; height: 200px; background-color: #ccc;"
  ></div>
  ```
  JavaScript on the page will look for any elements with `data-map-latitude` and `data-map-longitude` and, if it finds any, will load Leaflet and convert those elements into maps centered on that location. The default zoom level will be 12, or you can set a `data-map-zoom` attribute to customize this.

  ## Development

  To run the tests:

      uv run pytest

  To run tests against Datasette pre-1.0:

      uv run --with 'datasette<1.0' pytest

  To run tests against Datasette 1.0 and higher:

      uv run --with 'datasette>=1.0a26' pytest

project_urls:
  {"CI": "https://github.com/dogsheep/beta/actions", "Changelog": "https://github.com/dogsheep/beta/releases", "Homepage": "https://github.com/dogsheep/beta", "Issues": "https://github.com/dogsheep/beta/issues"}


name: dogsheep-photos
summary: Save details of your photos to a SQLite database and upload them to S3
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/dogsheep/dogsheep-photos
license: Apache License, Version 2.0
package_url: https://pypi.org/project/dogsheep-photos/
project_url: https://pypi.org/project/dogsheep-photos/
project_urls: {"Homepage": "https://github.com/dogsheep/dogsheep-photos"}
release_url: https://pypi.org/project/dogsheep-photos/0.4.1/
version: 0.4.1
description:
  # dogsheep-photos

  [![PyPI](https://img.shields.io/pypi/v/dogsheep-photos.svg)](https://pypi.org/project/dogsheep-photos/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/dogsheep-photos?include_prereleases&label=changelog)](https://github.com/dogsheep/dogsheep-photos/releases)
  [![CircleCI](https://circleci.com/gh/dogsheep/dogsheep-photos.svg?style=svg)](https://circleci.com/gh/dogsheep/dogsheep-photos)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/dogsheep-photos/blob/master/LICENSE)

  Save details of your photos to a SQLite database and upload them to S3.

  See [Using SQL to find my best photo of a pelican according to Apple Photos](https://simonwillison.net/2020/May/21/apple-photos-sqlite/) for background information on this project.

  ## What these tools do

  These tools are a work-in-progress mechanism for taking full ownership of your photos. The core idea is to help implement the following:

  * Every photo you have taken lives in a single, private Amazon S3 bucket
  * You have a single SQLite database file which stores metadata about those photos - potentially pulled from multiple different places. This may include EXIF data, Apple Photos, the results of running machine learning APIs against photos and much more besides.
  * You can then use [Datasette](https://github.com/simonw/datasette) to explore your own photos.

  I'm a heavy user of Apple Photos so the initial releases of this tool will have a bias towards that, but ideally I would like a subset of these tools to be useful to people no matter which core photo solution they are using.

  ## Installation

      $ pip install dogsheep-photos

  ## Authentication (if using S3)

  If you want to use S3 to store your photos, you will need to first create S3 credentials for a new, dedicated bucket.

  This is a big pain. Here's [how I did it](https://github.com/dogsheep/dogsheep-photos/issues/4).

  Run this command and paste in your credentials. You will need three values: the name of your S3 bucket, your Access key ID and your Secret access key.

      $ dogsheep-photos s3-auth

  This will create a file called `auth.json` in your current directory containing the required values. To save the file at a different path or filename, use the `--auth=myauth.json` option.

  ## Uploading photos

  Run this command to upload every photo in a specific directory to your S3 bucket:

      $ dogsheep-photos upload photos.db \
          ~/Pictures/Photos\ Library.photoslibrary/original

  The command will only upload photos that have not yet been uploaded, based on their sha256 hash.

  `photos.db` will be created with an `uploads` table containing details of which files were uploaded.

  To see what the command would do without uploading any files, use the `--dry-run` option.

  The sha256 hash of the photo contents will be used as the name of the file in the bucket, with an extension matching the type of file. This is an implementation of the [Content addressable storage](https://en.wikipedia.org/wiki/Content-addressable_storage) pattern.

  ## Importing Apple Photos metadata

  The `apple-photos` command imports metadata from your Apple Photos library.

      $ photo-to-sqlite apple-photos photos.db

  Imported metadata includes places, people, albums, quality scores and machine learning labels for the photo contents.

  ## Creating a subset database

  You can create a new, subset database of photos using the `create-subset` command.

  This is useful for creating a shareable SQLite database that only contains metadata for a selected set of photos.

  Since photo metadata contains latitude and longitude you may not want to share a database that includes photos taken at your home address.

  `create-subset` takes three arguments: an existing database file created using the `apple-photos` command, the name of the new, shareable database file you would like to create and a SQL query that returns the `sha256` hash values of the photos you would like to include in that database.

  For example, here's how to create a shareable database of just the photos that have been added to albums containing the word "Public":

      $ dogsheep-photos create-subset \
          photos.db \
          public.db \
          "select sha256 from apple_photos where albums like '%Public%'"

  ## Serving photos locally with datasette-media

  If you don't want to upload your photos to S3 but you still want to browse them using Datasette you can do so using the [datasette-media](https://github.com/simonw/datasette-media) plugin. This plugin adds the ability to serve images and other static files directly from disk, configured using a SQL query.

  To use it, first install Datasette and the plugin:

      $ pip install datasette datasette-media

  If any of your photos are `.HEIC` images taken by an iPhone you should also install the optional `pyheif` dependency:

      $ pip install pyheif

  Now create a `metadata.yaml` file configuring the plugin:

  ```yaml
  plugins:
    datasette-media:
      thumbnail:
        sql: |-
          select path as filepath, 200 as resize_height from apple_photos where uuid = :key
      large:
        sql: |-
          select path as filepath, 1024 as resize_height from apple_photos where uuid = :key
  ```
  This will configure two URL endpoins - one for 200 pixel high thumbnails and one for 1024 pixel hight larger images.

  Create your `photos.db` database using the `apple-photos` command, then run Datasette like this:

      $ datasette -m metadata.yaml

  Your photos will be served on URLs that look like this:

      http://127.0.0.1:8001/-/media/thumbnail/F4469918-13F3-43D8-9EC1-734C0E6B60AD
      http://127.0.0.1:8001/-/media/large/F4469918-13F3-43D8-9EC1-734C0E6B60AD

  You can find the UUIDs for use in these URLs by running `select uuid from photos_with_apple_metadata`.

  ### Displaying images using datasette-json-html

  If you are using `datasette-media` to serve photos you can include images directly in Datasette query results using the [datasette-json-html](https://github.com/simonw/datasette-json-html) plugin.

  Run `pip install datasette-json-html` to install the plugin, then use queries like this to view your images:

  ```sql
  select
      json_object(
          'img_src',
          '/-/media/thumbnail/' || uuid
      ) as photo,
      uuid,
      date
  from
      apple_photos
  order by
      date desc
  limit 10;
  ```
  The `photo` column returned by this query should render as image tags that display the correct images.

  ### Displaying images using custom template pages

  Datasette's [custom pages](https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-pages) feature lets you create custom pages for a Datasette instance by dropping HTML templates into a `templates/pages` directory and then running Datasette using `datasette --template-dir=templates/`.

  You can combine that ability with the [datasette-template-sql](https://github.com/simonw/datasette-template-sql) plugin to create custom template pages that directly display photos served by `datasette-media`.

  Install the plugin using `pip install datasette-template-sql`.

  Create a `templates/pages` folder and add the following files:

  `recent-photos.html`
  ```html+jinja
  <h1>Recent photos</h1>

  <div>
  {% for photo in sql("select * from apple_photos order by date desc limit 20") %}
      <img src="/-/media/photo/{{ photo['uuid'] }}">
  {% endfor %}
  </div>
  ```
  `random-photos.html`
  ```html+jinja
  <h1>Random photos</h1>

  <div>
  {% for photo in sql("with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 20") %}
      <img src="/-/media/photo/{{ photo['uuid'] }}">
  {% endfor %}
  </div>
  ```
  Now run Datasette like this:

      $ datasette photos.db -m metadata.yaml --template-dir=templates/

  Visiting `http://localhost:8001/recent-photos` will display 20 recent photos. Visiting `http://localhost:8001/random-photos` will display 20 photos randomly selected from your 5,000 most recent.

requires_dist:
  ["sqlite-utils (>=2.7)", "boto3 (>=1.12.41)", "osxphotos (>=0.28.13) ; sys_platform == \"darwin\"", "pytest ; extra == 'test'"]


name: download-tiles
summary: Download map tiles and store them in an MBTiles database
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/download-tiles
license: Apache License, Version 2.0
package_url: https://pypi.org/project/download-tiles/
project_url: https://pypi.org/project/download-tiles/
release_url: https://pypi.org/project/download-tiles/0.4.1/
requires_python: >=3.6
version: 0.4.1
description:
  # download-tiles

  [![PyPI](https://img.shields.io/pypi/v/download-tiles.svg)](https://pypi.org/project/download-tiles/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/download-tiles?include_prereleases&label=changelog)](https://github.com/simonw/download-tiles/releases)
  [![Tests](https://github.com/simonw/download-tiles/workflows/Test/badge.svg)](https://github.com/simonw/download-tiles/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/download-tiles/blob/master/LICENSE)

  Download map tiles and store them in an MBTiles database

  ## Installation

  Install this tool using `pip`:

      $ pip install download-tiles

  ## Usage

  This tool downloads tiles from a specified [TMS (Tile Map Server)](https://wiki.openstreetmap.org/wiki/TMS) server for a specified bounding box and range of zoom levels and stores those tiles in a MBTiles SQLite database. It is a command-line wrapper around the [Landez](https://github.com/makinacorpus/landez) Python libary.

  **Please use this tool responsibly**. Consult the usage policies of the tile servers you are interacting with, for example the [OpenStreetMap Tile Usage Policy](https://operations.osmfoundation.org/policies/tiles/).

  Running the following will download zoom levels 0-3 of OpenStreetMap, 85 tiles total, and store them in a SQLite database called `world.mbtiles`:

      download-tiles world.mbtiles

  You can customize which tile and zoom levels are downloaded using command options:

  `--zoom-levels=0-3` or `-z=0-3`

  The different zoom levels to download. Specify a single number, e.g. `15`, or a range of numbers e.g. `0-4`. Be careful with this setting as you can easily go over the limits requested by the underlying tile server.

  `--bbox=3.9,-6.3,14.5,10.2` or `-b=3.9,-6.3,14.5,10.2`

  The bounding box to fetch. Should be specified as `min-lon,min-lat,max-lon,max-lat`. You can use [bboxfinder.com](http://bboxfinder.com/) to find these for different areas.

  `--city=london` or `--country=madagascar`

  These options can be used instead of `--bbox`. The city or country specified will be looked up using the [Nominatum API](https://nominatim.org/release-docs/latest/api/Search/) and used to derive a bounding box.

  `--show-bbox`

  Use this option to output the bounding box that was retrieved for the `--city` or `--country` without downloading any tiles.

  `--name=Name`

  A name for this tile collection, used for the `name` field in the `metadata` table. If not specified a UUID will be used, or if you used `--city` or `--country` the name will be set to the full name of that place.

  `--attribution="Attribution string"`

  Attribution string to bake into the `metadata` table. This will default to `© OpenStreetMap contributors` unless you use `--tiles-url` to specify an alternative tile server, in which case you should specify a custom attribution string.

  You can use the `--attribution=osm` shortcut to specify the `© OpenStreetMap contributors` value without having to type it out in full.

  `--tiles-url=https://...`

  The tile server URL to use. This should include `{z}` and `{x}` and `{y}` specifiers, and can optionally include `{s}` for subdomains.

  The default URL used here is for OpenStreetMap, `http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png`

  `--tiles-subdomains=a,b,c`

  A comma-separated list of subdomains to use for the `{s}` parameter.

  `--verbose`

  Use this option to turn on verbose logging.

  `--cache-dir=/tmp/tiles`

  Provide a directory to cache downloaded tiles between runs. This can be useful if you are worried you might not have used the correct options for the bounding box or zoom levels.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd download-tiles
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/download-tiles/actions", "Changelog": "https://github.com/simonw/download-tiles/releases", "Homepage": "https://github.com/simonw/download-tiles", "Issues": "https://github.com/simonw/download-tiles/issues"}

requires_dist:
  ["click", "requests", "landez (==2.5.0)", "pytest ; extra == 'test'", "requests-mock ; extra == 'test'"]


name: evernote-to-sqlite
summary: Tools for converting Evernote content to SQLite
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/dogsheep/evernote-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/evernote-to-sqlite/
project_url: https://pypi.org/project/evernote-to-sqlite/
release_url: https://pypi.org/project/evernote-to-sqlite/0.3.2/
requires_dist: ["click", "sqlite-utils (>=3.0)", "pytest ; extra == 'test'"]
version: 0.3.2
description:
  # evernote-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/evernote-to-sqlite.svg)](https://pypi.org/project/evernote-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/evernote-to-sqlite?include_prereleases&label=changelog)](https://github.com/dogsheep/evernote-to-sqlite/releases)
  [![Tests](https://github.com/dogsheep/evernote-to-sqlite/workflows/Test/badge.svg)](https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE)

  Tools for converting Evernote content to SQLite. See [Building an Evernote to SQLite exporter](https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/) for background on this project.

  ## Installation

  Install this tool using `pip`:

      $ pip install evernote-to-sqlite

  ## Usage

  Currently the only available command is `evernote-to-sqlite enex`, which converts Evernote's ENEX export files into a SQLite database.

  You can create [an ENEX export](https://help.evernote.com/hc/en-us/articles/209005557-Export-notes-and-notebooks-as-ENEX-or-HTML) in the Evernote desktop application by selecting some notes (or all of your notes) and using the `File -> Export Notes...` menu option.

  This used to be able to export everything in one go, but it looks like more recent Evernote versions only allow exporting up to fifty notes at a time, or let you export an entire notebook by right-clicking on the notebook and selecting "Export notebook...".

  You can convert that file to SQLite like so:

      $ evernote-to-sqlite enex evernote.db MyNotes.enex

  This will display a progress bar and create a SQLite database file called `evernote.db`.

  ### Limitations

  Unfortunately the ENEX export format does not include a unique identifier for each note. This means you cannot use this tool to re-import notes after they have been updated - you should consider this tool to be a one-time transformation of an ENEX file into an equivalent SQLite database.

  ENEX exports also do not include details of which notebook a note belongs to.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd evernote-to-sqlite
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/dogsheep/evernote-to-sqlite/actions", "Changelog": "https://github.com/dogsheep/evernote-to-sqlite/releases", "Homepage": "https://github.com/dogsheep/evernote-to-sqlite", "Issues": "https://github.com/dogsheep/evernote-to-sqlite/issues"}


name: fec-to-sqlite
summary: Save FEC campaign finance data to a SQLite database
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/dogsheep/fec-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/fec-to-sqlite/
project_url: https://pypi.org/project/fec-to-sqlite/
project_urls: {"Homepage": "https://github.com/dogsheep/fec-to-sqlite"}
release_url: https://pypi.org/project/fec-to-sqlite/0.2/
requires_dist: ["sqlite-utils", "click", "requests", "fecfile", "tqdm", "pytest ; extra == 'test'"]
version: 0.2
description:
  # fec-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/fec-to-sqlite.svg)](https://pypi.org/project/fec-to-sqlite/)
  [![CircleCI](https://circleci.com/gh/simonw/fec-to-sqlite.svg?style=svg)](https://circleci.com/gh/simonw/fec-to-sqlite)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/fec-to-sqlite/blob/master/LICENSE)

  Create a SQLite database using FEC campaign contributions data.

  This tool builds on [fecfile](https://github.com/esonderegger/) by Evan Sonderegger.

  ## How to install

      $ pip install fec-to-sqlite

  ## Usage

      $ fec-to-sqlite filings filings.db 1146148

  This fetches the filing with ID `1146148` and stores it in tables in a SQLite database called `filings.db`. It will create any tables it needs.

  You can pass more than one filing ID, separated by spaces.


name: genome-to-sqlite
summary: Import your genome into a SQLite database
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/dogsheep/genome-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/genome-to-sqlite/
project_url: https://pypi.org/project/genome-to-sqlite/
project_urls: {"Homepage": "https://github.com/dogsheep/genome-to-sqlite"}
release_url: https://pypi.org/project/genome-to-sqlite/0.1/
requires_dist: ["sqlite-utils", "pytest ; extra == 'test'"]
version: 0.1
description:
  # genome-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/genome-to-sqlite.svg)](https://pypi.org/project/genome-to-sqlite/)
  [![CircleCI](https://circleci.com/gh/dogsheep/genome-to-sqlite.svg?style=svg)](https://circleci.com/gh/dogsheep/genome-to-sqlite)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/genome-to-sqlite/blob/master/LICENSE)

  Import your genome into a SQLite database.

  ## How to install

      $ pip install genome-to-sqlite

  ## How to use

  First, export your genome. This tool has only been tested against 23andMe so far. You can request an export of your genome from https://you.23andme.com/tools/data/download/

  Now you can convert the resulting `export.zip` file to SQLite like so:

      $ genome-to-sqlite export.zip genome.db

  A progress bar will be displayed. You can disable this using `--silent`.

  ```
  Importing genome  [#----------------]    5%  00:01:33
  ```

  You can explore the resulting data using [Datasette](https://datasette.readthedocs.io/) like this:

      $ datasette genome.db --config facet_time_limit_ms:1000

  Bumping up the facet time limit is useful in order to enable faceting by chromosome:

  http://127.0.0.1:8001/genome/genome?_facet=chromosome&_sort=position


name: geocode-sqlite
summary: Geocode rows from a SQLite table
classifiers: []
author: Chris Amico
description_content_type: text/markdown
home_page: https://github.com/eyeseast/geocode-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/geocode-sqlite/
project_url: https://pypi.org/project/geocode-sqlite/
release_url: https://pypi.org/project/geocode-sqlite/0.8.2/
version: 0.8.2
description:
  # geocode-sqlite

  [![PyPI](https://img.shields.io/pypi/v/geocode-sqlite.svg)](https://pypi.org/project/geocode-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/geocode-sqlite?include_prereleases&label=changelog)](https://github.com/eyeseast/geocode-sqlite/releases)
  [![Tests](https://github.com/eyeseast/geocode-sqlite/workflows/Test/badge.svg)](https://github.com/eyeseast/geocode-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/geocode-sqlite/blob/master/LICENSE)

  Geocode rows from a SQLite table

  ## Installation

  Install this tool using `pip` or `pipx`:

  ```sh
  # install inside a virtualenv
  pip install geocode-sqlite

  # install globally
  pipx install geocode-sqlite
  ```

  ## Usage

  Let's say you have a spreadsheet with addresses in it, and you'd like to map those locations.
  First, create a SQLite database and insert rows from that spreadsheet using `sqlite-utils`.

  ```sh
  sqlite-utils insert data.db data data.csv --csv
  ```

  Now, geocode it using OpenStreetMap's Nominatim geocoder.

  ```sh
  geocode-sqlite nominatim data.db data \
   --location="{address}, {city}, {state} {zip}" \
   --delay=1 \
   --user-agent="this-is-me"
  ```

  In the command above, you're using Nominatim, which is free and only asks for a unique user agent (`--user-agent`).

  This will connect to a database (`data.db`) and read all rows from the table `data` (skipping any that already
  have both a `latitude` and `longitude` column filled).

  You're also telling the geocoder how to extract a location query (`--location`) from a row of data, using Python's
  built-in string formatting, and setting a rate limit (`--delay`) of one request per second.

  For each row where geocoding succeeds, `latitude` and `longitude` will be populated. If you hit an error, or a rate limit,
  run the same query and pick up where you left off.

  The resulting table layout can be visualized with [datasette-cluster-map](https://datasette.io/plugins/datasette-cluster-map).

  Under the hood, this package uses the excellent [geopy](https://geopy.readthedocs.io/en/latest/) library, which is stable and thoroughly road-tested. If you need help understanding a particular geocoder's options, consult [geopy's documentation](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders).

  ### Supported Geocoders

  The CLI currently supports these geocoders:

  - `bing`
  - `googlev3`
  - `mapquest` (and `open-mapquest`)
  - `mapbox`
  - `nominatim`
  - `opencage`

  #### Adding new geocoders

  1. Open an issue with the name of the geocoding service as the ticket title ([example](https://github.com/eyeseast/geocode-sqlite/issues/35)). Put any noteworthy implementation details in the ticket body, like where to get an API key if one is required.
  2. Fork the repo and add a geocoder.
  3. Add an example to the `Makefile`. Add tests if there's new shared functionality.

  ### Common arguments and options

  Each geocoder needs to know where to find the data it's working with. These are the first two arguments:

  - `database`: a path to a SQLite file, which must already exist
  - `table`: the name of a table, in that database, which exists and has data to geocode

  From there, we have a set of options passed to every geocoder:

  - `location`: a [string format](https://docs.python.org/3/library/stdtypes.html#str.format) that will be expanded with each row to build a full query, to be geocoded
  - `delay`: a delay between each call (some services require this)
  - `latitude`: latitude column name
  - `longitude`: longitude column name
  - `geojson`: store results as GeoJSON, instead of in latitude and longitude columns
  - `spatialite`: store results in a SpatiaLite geometry column, instead of in latitude and longitude columns
  - `raw`: store raw geocoding results in a JSON column

  Each geocoder takes additional, specific arguments beyond these, such as API keys. Again, [geopy's documentation](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders) is an excellent resource.

  ## Using SpatiaLite

  The `--spatialite` flag will store results in a [geometry column](https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/cookbook_topics.adminstration.html#topic_TABLE_to_SpatialTable), instead of `latitude` and `longitude` columns. This is useful if you're doing other GIS operations, such as using a [spatial index](https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex). See the [SpatiaLite cookbook](https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/index.html) and [functions list](https://www.gaia-gis.it/gaia-sins/spatialite-sql-latest.html) for more of what's possible.

  ## Capturing additional geocoding data

  Geocoding services typically return more data than just coordinates. This might include accuracy, normalized addresses or other context. This can be captured using the `--raw` flag. By default, this will add a `raw` column and store the full geocoding response as JSON. If you want to rename that column, pass a value, like `--raw custom_raw`.

  The shape of this response object will vary between services. You can query specific values using [SQLite's built-in JSON functions](https://www.sqlite.org/json1.html). For example, this will work with Google's geocoder:

  ```sql
  select
    json_extract(raw, '$.formatted_address') as address,
    json_extract(raw, '$.geometry.location_type') as location_type
  from
    innout_test
  ```

  Check each geocoding service's documentation for what's included in the response.

  ## Python API

  The command line interface aims to support the most common options for each geocoder. For more fine-grained control, use the Python API.

  As with the CLI, this assumes you already have a SQLite database and a table of location data.

  ```python
  from geocode_sqlite import geocode_table
  from geopy.geocoders import Nominatim

  # create a geocoder instance, with some extra options
  nominatim = Nominatim(user_agent="this-is-me", domain="nominatim.local.dev", scheme="http")

  # assuming our database is in the same directory
  count = geocode_table("data.db", "data", query_template="{address}, {city}, {state} {zip}")

  # when it's done
  print(f"Geocoded {count} rows")
  ```

  Any [geopy geocoder](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders) can be used with the Python API.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

  ```sh
  cd geocode-sqlite
  python -m venv .venv
  source .venv/bin/activate
  ```

  Or if you are using `pipenv`:

  ```sh
  pipenv shell
  ```

  Now install the dependencies and tests:

  ```sh
  pip install -e '.[test]'
  ```

  To run the tests:

  ```sh
  pytest
  ```

  Please remember that this library is mainly glue code between other well-tested projects, specifically: [click](https://click.palletsprojects.com/), [geopy](https://geopy.readthedocs.io/en/stable/) and [sqlite-utils](https://sqlite-utils.datasette.io/en/stable/). Tests should focus on making sure those parts fit together correctly. We can assume the parts themselves already work.

  To that end, there is a test geocoder included: `geocode_sqlite.testing.DummyGeocoder`. That geocoder works with an included dataset of In-N-Out Burger locations provided by [AllThePlaces](https://www.alltheplaces.xyz/). It works like a normal GeoPy geocoder, except it will only return results for In-N-Out locations using the included database.

project_urls:
  {"CI": "https://github.com/eyeseast/geocode-sqlite/actions", "Changelog": "https://github.com/eyeseast/geocode-sqlite/releases", "Homepage": "https://github.com/eyeseast/geocode-sqlite", "Issues": "https://github.com/eyeseast/geocode-sqlite/issues"}

requires_dist:
  ["click (>=7.0)", "sqlite-utils", "geopy", "requests", "pytest ; extra == 'test'", "geojson-to-sqlite ; extra == 'test'"]


name: geojson-to-sqlite
summary: CLI tool for converting GeoJSON to SQLite (optionally with SpatiaLite)
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/geojson-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/geojson-to-sqlite/
project_url: https://pypi.org/project/geojson-to-sqlite/
project_urls: {"Homepage": "https://github.com/simonw/geojson-to-sqlite"}
release_url: https://pypi.org/project/geojson-to-sqlite/1.1.1/
requires_dist: ["sqlite-utils (>=3.23)", "shapely", "pytest ; extra == 'test'", "dirty-equals ; extra == 'test'"]
version: 1.1.1
description:
  # geojson-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/geojson-to-sqlite.svg)](https://pypi.org/project/geojson-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/geojson-to-sqlite?include_prereleases&label=changelog)](https://github.com/simonw/geojson-to-sqlite/releases)
  [![Tests](https://github.com/simonw/geojson-to-sqlite/workflows/Test/badge.svg)](https://github.com/simonw/geojson-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/geojson-to-sqlite/blob/main/LICENSE)

  CLI tool for converting GeoJSON to SQLite (optionally with SpatiaLite)

  [RFC 7946: The GeoJSON Format](https://tools.ietf.org/html/rfc7946)

  ## How to install

      $ pip install geojson-to-sqlite

  ## How to use

  You can run this tool against a GeoJSON file like so:

      $ geojson-to-sqlite my.db features features.geojson

  This will load all of the features from the `features.geojson` file into a table called `features`.

  Each row will have a `geometry` column containing the feature geometry, and columns for each of the keys found in any `properties` attached to those features. (To bundle all properties into a single JSON object, use the `--properties` flag.)

  The table will be created the first time you run the command.

  On subsequent runs you can use the `--alter` option to add any new columns that are missing from the table.

  You can pass more than one GeoJSON file, in which case the contents of all of the files will be inserted into the same table.

  If your features have an `"id"` property it will be used as the primary key for the table. You can also use `--pk=PROPERTY` with the name of a different property to use that as the primary key instead. If you don't want to use the `"id"` as the primary key (maybe it contains duplicate values) you can use `--pk ''` to specify no primary key.

  Specifying a primary key also will allow you to upsert data into the rows instead of insert data into new rows.

  If no primary key is specified, a SQLite `rowid` column will be used.

  You can use `-` as the filename to import from standard input. For example:

      $ curl https://eric.clst.org/assets/wiki/uploads/Stuff/gz_2010_us_040_00_20m.json \
          | geojson-to-sqlite my.db states - --pk GEO_ID

  ## Using with SpatiaLite

  By default, the `geometry` column will contain JSON.

  If you have installed the [SpatiaLite](https://www.gaia-gis.it/fossil/libspatialite/index) module for SQLite you can instead import the geometry into a geospatially indexed column.

  You can do this using the `--spatialite` option, like so:

      $ geojson-to-sqlite my.db features features.geojson --spatialite

  The tool will search for the SpatiaLite module in the following locations:

  - `/usr/lib/x86_64-linux-gnu/mod_spatialite.so`
  - `/usr/local/lib/mod_spatialite.dylib`

  If you have installed the module in another location, you can use the `--spatialite_mod=xxx` option to specify where:

      $ geojson-to-sqlite my.db features features.geojson \
          --spatialite_mod=/usr/lib/mod_spatialite.dylib

  You can create a SpatiaLite spatial index on the `geometry` column using the `--spatial-index` option:

      $ geojson-to-sqlite my.db features features.geojson --spatial-index

  Using this option implies `--spatialite` so you do not need to add that.

  ## Streaming large datasets

  For large datasets, consider using newline-delimited JSON to stream features into the database without loading the entire feature collection into memory.

  For example, to load a day of earthquake reports from USGS:

      $ geojson-to-sqlite quakes.db quakes tests/quakes.ndjson \
        --nl --pk=id --spatialite

  When using newline-delimited JSON, tables will also be created from the first feature, instead of guessing types based on the first 100 features.

  If you want to use a larger subset of your data to guess column types (for example, if some fields are inconsistent) you can use [fiona](https://fiona.readthedocs.io/en/latest/cli.html) to collect features into a single collection.

      $ head tests/quakes.ndjson | fio collect | \
        geojson-to-sqlite quakes.db quakes - --spatialite

  This will take the first 10 lines from `tests/quakes.ndjson`, pass them to `fio collect`, which turns them into a single feature collection, and pass that, in turn, to `geojson-to-sqlite`.

  ## Using this with Datasette

  Databases created using this tool can be explored and published using [Datasette](https://datasette.readthedocs.io/).

  The Datasette documentation includes a section on [how to use it to browse SpatiaLite databases](https://datasette.readthedocs.io/en/stable/spatialite.html).

  The [datasette-leaflet-geojson](https://datasette.io/plugins/datasette-leaflet-geojson) plugin can be used to visualize columns containing GeoJSON geometries on a [Leaflet](https://leafletjs.com/) map.

  If you are using SpatiaLite you will need to output the geometry as GeoJSON in order for that plugin to work. You can do that using the SpaitaLite `AsGeoJSON()` function - something like this:

  ```sql
  select rowid, AsGeoJSON(geometry) from mytable limit 10
  ```

  The [datasette-geojson-map](https://datasette.io/plugins/datasette-geojson-map) is an alternative plugin which will automatically render SpatiaLite geometries as a Leaflet map on the corresponding table page, without needing you to call `AsGeoJSON(geometry)`.


name: git-history
summary: Tools for analyzing Git history using SQLite
classifiers: []
author: Simon Willison
description_content_type: text/markdown
dynamic: ["License-File"]
license_expression: Apache-2.0
license_files: ["LICENSE"]
package_url: https://pypi.org/project/git-history/
project_url: https://pypi.org/project/git-history/
release_url: https://pypi.org/project/git-history/0.8/
requires_dist: ["click", "GitPython", "sqlite-utils>=3.19"]
requires_python: >=3.10
version: 0.8
description:
  # git-history

  [![PyPI](https://img.shields.io/pypi/v/git-history.svg)](https://pypi.org/project/git-history/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/git-history?include_prereleases&label=changelog)](https://github.com/simonw/git-history/releases)
  [![Tests](https://github.com/simonw/git-history/workflows/Test/badge.svg)](https://github.com/simonw/git-history/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/git-history/blob/master/LICENSE)

  Tools for analyzing Git history using SQLite

  For background on this project see [git-history: a tool for analyzing scraped data collected using Git and SQLite](https://simonwillison.net/2021/Dec/7/git-history/).

  [Measuring traffic during the Half Moon Bay Pumpkin Festival](https://simonwillison.net/2022/Oct/19/measuring-traffic/) describes a project using this tool in detail.

  ## Installation

  Install this tool using `uv`:
  ```bash
  uv tool install git-history
  ```
  Or run it without installing it:
  ```bash
  uvx git-history --help
  ```

  ## Demos

  [git-history-demos.datasette.io](http://git-history-demos.datasette.io/) hosts three example databases created using this tool:

  - [pge-outages](https://git-history-demos.datasette.io/pge-outages) shows a history of PG&E (the electricity supplier) [outages](https://pgealerts.alerts.pge.com/outagecenter/), using data collected in [simonw/pge-outages](https://github.com/simonw/pge-outages) converted using [pge-outages.sh](https://github.com/simonw/git-history/blob/main/demos/pge-outages.sh)
  - [ca-fires](https://git-history-demos.datasette.io/ca-fires) shows a history of fires in California reported on [fire.ca.gov/incidents](https://www.fire.ca.gov/incidents/), from data in [simonw/ca-fires-history](https://github.com/simonw/ca-fires-history) converted using [ca-fires.sh](https://github.com/simonw/git-history/blob/main/demos/ca-fires.sh)

  The demos are deployed using [Datasette](https://datasette.io/) on [Google Cloud Run](https://cloud.google.com/run/) by [this GitHub Actions workflow](https://github.com/simonw/git-history/blob/main/.github/workflows/deploy-demos.yml).

  ## Usage

  This tool can be run against a Git repository that holds a file that contains JSON, CSV/TSV or some other format and which has multiple versions tracked in the Git history. Read [Git scraping: track changes over time by scraping to a Git repository](https://simonwillison.net/2020/Oct/9/git-scraping/) to understand how you might create such a repository.

  The `file` command analyzes the history of an individual file within the repository, and generates a SQLite database table that represents the different versions of that file over time.

  The file is assumed to contain multiple objects - for example, the results of scraping an electricity outage map or a CSV file full of records.

  Assuming you have a file called `incidents.json` that is a JSON array of objects, with multiple versions of that file recorded in a repository. Each version of that file might look something like this:

  ```json
  [
      {
          "IncidentID": "abc123",
          "Location": "Corner of 4th and Vermont",
          "Type": "fire"
      },
      {
          "IncidentID": "cde448",
          "Location": "555 West Example Drive",
          "Type": "medical"
      }
  ]
  ```

  Change directory into the GitHub repository in question and run the following:

      git-history file incidents.db incidents.json

  You can also analyze a remote Git repository by passing a URL to the `--repo` option. The repository will be cloned to a temporary directory before processing.

  Using an HTTPS URL:

      git-history file incidents.db incidents.json \
        --repo https://github.com/example/repo

  Using a git SSH URL:

      git-history file incidents.db incidents.json \
        --repo git@github.com:example/repo

  When using a remote URL, the filepath argument should be a relative path within the repository (e.g. `incidents.json` or `data/incidents.json`).

  This will create a new SQLite database in the `incidents.db` file with three tables:

  - `commits` containing a row for every commit, with a `hash` column, the `commit_at` date and a foreign key to a `namespace`.
  - `item` containing a row for every item in every version of the `filename.json` file - with an extra `_commit` column that is a foreign key back to the `commit` table.
  - `namespaces` containing a single row. This allows you to build multiple tables for different files, using the `--namespace` option described below.

  The database schema for this example will look like this:

  <!-- [[[cog
  import cog, json
  from git_history import cli
  from click.testing import CliRunner
  from tests.test_git_history import make_repo
  import sqlite_utils
  import tempfile, pathlib
  tmpdir = pathlib.Path(tempfile.mkdtemp())
  db_path = str(tmpdir / "data.db")
  make_repo(tmpdir)
  runner = CliRunner()
  result = runner.invoke(cli.cli, [
      "file", db_path, str(tmpdir / "repo" / "incidents.json"), "--repo", str(tmpdir / "repo")
  ])
  cog.out("```sql\n")
  cog.out(sqlite_utils.Database(db_path).schema)
  cog.out("\n```")
  ]]] -->
  ```sql
  CREATE TABLE [namespaces] (
     [id] INTEGER PRIMARY KEY,
     [name] TEXT
  );
  CREATE UNIQUE INDEX [idx_namespaces_name]
      ON [namespaces] ([name]);
  CREATE TABLE [commits] (
     [id] INTEGER PRIMARY KEY,
     [namespace] INTEGER REFERENCES [namespaces]([id]),
     [hash] TEXT,
     [commit_at] TEXT
  );
  CREATE UNIQUE INDEX [idx_commits_namespace_hash]
      ON [commits] ([namespace], [hash]);
  CREATE TABLE [item] (
     [IncidentID] TEXT,
     [Location] TEXT,
     [Type] TEXT,
     [_commit] INTEGER REFERENCES [commits]([id])
  );
  ```
  <!-- [[[end]]] -->

  If you have 10 historic versions of the `incidents.json` file and each one contains 30 incidents, you will end up with 10 * 30 = 300 rows in your `item` table.

  ### Track the history of individual items using IDs

  If your objects have a unique identifier - or multiple columns that together form a unique identifier - you can use the `--id` option to de-duplicate and track changes to each of those items over time.

  This provides a much more interesting way to apply this tool.

  If there is a unique identifier column called `IncidentID` you could run the following:

      git-history file incidents.db incidents.json --id IncidentID

  The database schema used here is very different from the one used without the `--id` option.

  If you have already imported history, the command will skip any commits that it has seen already and just process new ones. This means that even though an initial import could be slow subsequent imports should run a lot faster.

  This command will create six tables - `commits`, `item`, `item_version`, `columns`, `item_changed` and `namespaces`.

  Here's the full schema:

  <!-- [[[cog
  db_path2 = str(tmpdir / "data2.db")
  result = runner.invoke(cli.cli, [
      "file", db_path2, str(tmpdir / "repo" / "incidents.json"),
      "--repo", str(tmpdir / "repo"),
      "--id", "IncidentID"
  ])
  cog.out("```sql\n")
  cog.out(sqlite_utils.Database(db_path2).schema)
  cog.out("\n```")
  ]]] -->
  ```sql
  CREATE TABLE [namespaces] (
     [id] INTEGER PRIMARY KEY,
     [name] TEXT
  );
  CREATE UNIQUE INDEX [idx_namespaces_name]
      ON [namespaces] ([name]);
  CREATE TABLE [commits] (
     [id] INTEGER PRIMARY KEY,
     [namespace] INTEGER REFERENCES [namespaces]([id]),
     [hash] TEXT,
     [commit_at] TEXT
  );
  CREATE UNIQUE INDEX [idx_commits_namespace_hash]
      ON [commits] ([namespace], [hash]);
  CREATE TABLE [item] (
     [_id] INTEGER PRIMARY KEY,
     [_item_id] TEXT
  , [IncidentID] TEXT, [Location] TEXT, [Type] TEXT, [_commit] INTEGER);
  CREATE UNIQUE INDEX [idx_item__item_id]
      ON [item] ([_item_id]);
  CREATE TABLE [item_version] (
     [_id] INTEGER PRIMARY KEY,
     [_item] INTEGER REFERENCES [item]([_id]),
     [_version] INTEGER,
     [_commit] INTEGER REFERENCES [commits]([id]),
     [IncidentID] TEXT,
     [Location] TEXT,
     [Type] TEXT,
     [_item_full_hash] TEXT
  );
  CREATE TABLE [columns] (
     [id] INTEGER PRIMARY KEY,
     [namespace] INTEGER REFERENCES [namespaces]([id]),
     [name] TEXT
  );
  CREATE UNIQUE INDEX [idx_columns_namespace_name]
      ON [columns] ([namespace], [name]);
  CREATE TABLE [item_changed] (
     [item_version] INTEGER REFERENCES [item_version]([_id]),
     [column] INTEGER REFERENCES [columns]([id]),
     PRIMARY KEY ([item_version], [column])
  );
  CREATE VIEW item_version_detail AS select
    commits.commit_at as _commit_at,
    commits.hash as _commit_hash,
    item_version.*,
    (
      select json_group_array(name) from columns
      where id in (
        select column from item_changed
        where item_version = item_version._id
      )
  ) as _changed_columns
  from item_version
    join commits on commits.id = item_version._commit;
  CREATE INDEX [idx_item_version__item]
      ON [item_version] ([_item]);
  ```
  <!-- [[[end]]] -->

  #### item table

  The `item` table will contain the most recent version of each row, de-duplicated by ID, plus the following additional columns:

  - `_id` - a numeric integer primary key, used as a foreign key from the `item_version` table.
  - `_item_id` - a hash of the values of the columns specified using the `--id` option to the command. This is used for de-duplication when processing new versions.
  - `_commit` - a foreign key to the `commit` table, representing the most recent commit to modify this item.

  #### item_version table

  The `item_version` table will contain a row for each captured differing version of that item, plus the following columns:

  - `_id` - a numeric ID for the item version record.
  - `_item` - a foreign key to the `item` table.
  - `_version` - the numeric version number, starting at 1 and incrementing for each captured version.
  - `_commit` - a foreign key to the `commit` table.
  - `_item_full_hash` - a hash of this version of the item. This is used internally by the tool to identify items that have changed between commits.

  The other columns in this table represent columns in the original data that have changed since the previous version. If the value has not changed, it will be represented by a `null`.

  If a value was previously set but has been changed back to `null` it will still be represented as `null` in the `item_version` row. You can identify these using the `item_changed` many-to-many table described below.

  You can use the `--full-versions` option to store full copies of the item at each version, rather than just storing the columns that have changed.

  #### item_version_detail view

  This SQL view joins `item_version` against `commits` to add three further columns: `_commit_at` with the date of the commit, and `_commit_hash` with the Git commit hash.

  #### item_changed

  This many-to-many table indicates exactly which columns were changed in an `item_version`.

  - `item_version` is a foreign key to a row in the `item_version` table.
  - `column` is a foreign key to a row in the `columns` table.

  This table with have the largest number of rows, which is why it stores just two integers in order to save space.

  #### columns

  The `columns` table stores column names. It is referenced by `item_changed`.

  - `id` - an integer ID.
  - `name` - the name of the column.
  - `namespace` - a foreign key to `namespaces`, for if multiple file histories are sharing the same database.

  #### Reserved column names

  <!-- [[[cog
  from git_history.utils import RESERVED
  cog.out("Note that ")
  cog.out(", ".join("`{}`".format(r) for r in RESERVED))
  cog.out(" are considered reserved column names for the purposes of this tool.")
  ]]] -->
  Note that `_id`, `_item_full_hash`, `_item`, `_item_id`, `_version`, `_commit`, `_item_id`, `_commit_at`, `_commit_hash`, `_changed_columns`, `rowid` are considered reserved column names for the purposes of this tool.
  <!-- [[[end]]] -->

  If your data contains any of these they will be renamed to add a trailing underscore, for example `_id_`, `_item_`, `_version_`, to avoid clashing with the reserved columns.

  If you have a column with a name such as `_commit_` it will be renamed too, adding an additional trailing underscore, so `_commit_` becomes `_commit__` and `_commit__` becomes `_commit___`.

  ### Additional options

  - `--repo DIRECTORY_OR_URL` - the path to the Git repository, or a URL to clone. If not specified, uses the current working directory. URLs can be HTTPS (e.g. `https://github.com/user/repo`) or SSH (e.g. `git@github.com:user/repo`).
  - `--branch TEXT` - the Git branch to analyze - defaults to `main`.
  - `--id TEXT` - as described above: pass one or more columns that uniquely identify a record, so that changes to that record can be calculated over time.
  - `--full-versions` - instead of recording just the columns that have changed in the `item_version` table record a full copy of each version of theh item.
  - `--ignore TEXT` - one or more columns to ignore - they will not be included in the resulting database.
  - `--csv` - treat the data is CSV or TSV rather than JSON, and attempt to guess the correct dialect
  - `--dialect` - use a spcific CSV dialect. Options are `excel`, `excel-tab` and `unix` - see [the Python CSV documentation](https://docs.python.org/3/library/csv.html#csv.excel) for details.
  - `--encoding` - character encoding to use when reading CSV files. Defaults to `utf-8`. Use this for files encoded in other formats such as `latin-1` or `utf-16`.
  - `--skip TEXT` - one or more full Git commit hashes that should be skipped. You can use this if some of the data in your revision history is corrupted in a way that prevents this tool from working.
  - `--start-at TEXT` - skip commits prior to the specified commit hash.
  - `--start-after TEXT` - skip commits up to and including the specified commit hash, then start processing from the following commit.
  - `--convert TEXT` - custom Python code for a conversion, described below.
  - `--import TEXT` - additional Python modules to import for `--convert`.
  - `--ignore-duplicate-ids` - if a single version of a file has the same ID in it more than once, the tool will exit with an error. Use this option to ignore this and instead pick just the first of the two duplicates.
  - `--namespace TEXT` - use this if you wish to include the history of multiple different files in the same database. The default is `item` but you can set it to something else, which will produce tables with names like `yournamespace` and `yournamespace_version`.
  - `--wal` - Enable WAL mode on the created database file. Use this if you plan to run queries against the database while `git-history` is creating it.
  - `--silent` - don't show the progress bar.

  ### CSV and TSV data

  If the data in your repository is a CSV or TSV file you can process it by adding the `--csv` option. This will attempt to detect which delimiter is used by the file, so the same option works for both comma- and tab-separated values.

      git-history file trees.db trees.csv --id TreeID

  You can also specify the CSV dialect using the `--dialect` option.

  ### Custom conversions using --convert

  If your data is not already either CSV/TSV or a flat JSON array, you can reshape it using the `--convert` option.

  The format needed by this tool is an array of dictionaries, as demonstrated by the `incidents.json` example above.

  If your data does not fit this shape, you can provide a snippet of Python code to converts the on-disk content of each stored file into a Python list of dictionaries.

  For example, if your stored files each look like this:

  ```json
  {
      "incidents": [
          {
              "id": "552",
              "name": "Hawthorne Fire",
              "engines": 3
          },
          {
              "id": "556",
              "name": "Merlin Fire",
              "engines": 1
          }
      ]
  }
  ```
  You could use the following Python snippet to convert them to the required format:

  ```python
  json.loads(content)["incidents"]
  ```
  (The `json` module is exposed to your custom function by default.)

  You would then run the tool like this:

      git-history file database.db incidents.json \
        --id id \
        --convert 'json.loads(content)["incidents"]'

  The `content` variable is always a `bytes` object representing the content of the file at a specific moment in the repository's history.

  You can import additional modules using `--import`. This example shows how you could read a CSV file that uses `;` as the delimiter:

      git-history file trees.db ../sf-tree-history/Street_Tree_List.csv \
        --repo ../sf-tree-history \
        --import csv \
        --import io \
        --convert '
          fp = io.StringIO(content.decode("utf-8"))
          return list(csv.DictReader(fp, delimiter=";"))
          ' \
        --id TreeID

  You can import nested modules such as [ElementTree](https://docs.python.org/3/library/xml.etree.elementtree.html)  using `--import xml.etree.ElementTree`, then refer to them in your function body as `xml.etree.ElementTree`. For example, if your tracked data was in an `items.xml` file that looked like this:

  ```xml
  <items>
    <item id="1" name="One" />
    <item id="2" name="Two" />
    <item id="3" name="Three" />
  </item>
  ```
  You could load it using the following `--convert` script:
  ```
  git-history file items.xml --convert '
  tree = xml.etree.ElementTree.fromstring(content)
  return [el.attrib for el in tree.iter("item")]
  ' --import xml.etree.ElementTree --id id
  ```

  If your Python code spans more than one line it needs to include a `return` statement.

  You can also use Python generators in your `--convert` code, for example:

      git-history file stats.db package-stats/stats.json \
          --repo package-stats \
          --convert '
          data = json.loads(content)
          for key, counts in data.items():
              for date, count in counts.items():
                  yield {
                      "package": key,
                      "date": date,
                      "count": count
                  }
          ' --id package --id date

  This conversion function expects data that looks like this:

  ```json
  {
      "airtable-export": {
          "2021-05-18": 66,
          "2021-05-19": 60,
          "2021-05-20": 87
      }
  }
  ```

  ## Development

  To contribute to this tool, first checkout the code. You can run the tests using `uv run`:
  ```bash
  cd git-history
  uv run pytest
  ```
  And run your local development copy of the tool like this
  ```bash
  uv run git-history --help
  ```
  To update the schema examples in this README file:
  ```bash
  uv run cog -r README.md
  ```

project_urls:
  {"CI": "https://github.com/simonw/git-history/actions", "Changelog": "https://github.com/simonw/git-history/releases", "Homepage": "https://github.com/simonw/git-history", "Issues": "https://github.com/simonw/git-history/issues"}


