Running the Examples

ElectricSQL provides a number of example applications on GitHub at github.com/electric-sql/examples. Some of which you can also see on our demos page.

These examples are configured to connect (when they’re online) to a sync service. By default, they use an existing cloud sync service, so you don’t have to run or provision anything. However, you have the option of setting up and using your own cloud sync service and/or running the backend stack yourself using our open source code.

This page walks through the options, so you can run the examples however you prefer.

Option 1 — use the default sync service

The simplest way is just to run the example application using the pre-configured sync service. See the instructions in the README.md but you can usually run something like:

yarn
yarn start

This will run the app locally, connecting to an existing cloud sync service that’s already been configured with the right data model and any seed data.

Option 2 — connect to your own sync service

Sign up to ElectricSQL and follow the instructions to create a new app. After the system provisions the infrastructure, you’ll be able to view the connection details for your sync service and Postgres database. Copy these somewhere safe (like a password manager).

Install the CLI. For example using Homebrew, or see the install guide for more options:

brew install electric-sql/tap/electric

Login on the command line (using the email and password you signed up with):

electric auth login EMAIL

Make sure you’re in the root of your example folder and run (replacing APP with the app identifier from your sync service connection details you copied earlier):

electric config update --app APP

This updates the ./electric.json config file and your migrations bundle to use the new app identifier.

You can now write migrations to change your schema, for example:

electric migrations new "drop items and create kv"

This generates a subfolder at ./migrations/*_drop_items_and_create_kv/ containing a migration.sql file. Open this file in your text editor and write the SQL DDL statements you want to create and alter your database schema. For example:

DROP TABLE IF EXISTS items;

CREATE TABLE IF NOT EXISTS kv (
  key TEXT PRIMARY KEY NOT NULL
  value TEXT
) WITHOUT ROWID;

Build your migrations:

electric build

Upload to your new cloud service:

electric sync

If you now run your app, it will apply the migrations to use your database schema and load the configuration in electric.json to connect to your cloud sync service.

Option 3 — run the backend locally

So far we’ve been using the hosted cloud sync service. However, ElectricSQL is open source software and you can also run the backend database and sync service components yourself.

Make sure you have make, Docker and at least Elixir 1.14 compiled with Erlang 24 installed.

Clone the electric-sql/electric repo:

git clone https://github.com/electric-sql/electric
cd electric

Fetch deps and compile:

make deps compile

Run the dependencies using:

make start_dev_env

Run the tests:

make tests

Develop using:

make shell

This runs active-active replication with Postgres over logical replication and exposes a protocol buffers API over web sockets on localhost:5133.

For example to write some data into one of the Postgres instances:

docker exec -it -e PGPASSWORD=password electric_db_a_1 psql -h 127.0.0.1 -U electric -d electric

There’s also a second instance, electric-db_b_1, if you want to see data being replicated between them.

Note that you can tear down all the containers with:

make stop_dev_env

Configure your application

Back in the root of your application repo, use the CLI to set the replication option in your electric.json:

electric config update --replication-disable-ssl --replication-host localhost --replication-port 5133

If you now run your application, you can change data using the GUI and monitor your terminal output to watch data flowing through the backend services. Also you can write data into Postgres and see it sync into the app.

Running the release or docker container

The Electric application is configured using environment variables. Everything that doesn’t have a default is required to run.

Variable Default Description
VAXINE_HOST   Host of Vaxine instance to connect to
VAXINE_API_PORT 8087 Port for the regular DB API on Vaxine instance
VAXINE_REPLICATION_PORT 8088 Port for the replication API on Vaxine instance
VAXINE_CONNECTION_TIMEOUT 5000 (ms) Timeout waiting while connecting to a Vaxine instance
     
ELECTRIC_HOST   Host of this electric instance for the reverse connection from Postgres. It has to be accessible from postgres instances listed in the CONNECTORS
CONNECTORS "" Semicolon-separated list of Postgres connection strings for PG instances that will be part of the cluster
     
POSTGRES_REPLICATION_PORT 5433 Port for connections from PG instances as replication followers
STATUS_PORT 5050 Port to expose health and status API endpoint
WEBSOCKET_PORT 5133 Port to expose the /ws path for the replication over the websocket
     
OFFSET_STORAGE_FILE ./offset_storage_data.dat Path to the file storing the mapping between connected instances and offsets in Vaxine WAL. Should be persisted between Electric restarts.
     
MIGRATIONS_DIR   Directory to read the migration SQL files from (see below)
MIGRATIONS_FILE_NAME_SUFFIX /postgres.sql Suffix that is appended to the migration name when looking for the migration file
     
SATELLITE_AUTH_SIGNING_KEY "" Authentication token signing/validation secret key. See below.
SATELLITE_AUTH_SIGNING_ISS "" Cluster ID which acts as the issuer for the authentication JWT. See below.
     
GLOBAL_CLUSTER_ID   Identifier of the cluster within the Electric cloud. When running locally, you can use any string
ELECTRIC_INSTANCE_ID   Unique identifier of this Electric instance when running in the cluster. When running locally, you can use any string
ELECTRIC_REGIONAL_ID   Shared identifier for multiple Electric instance that connect to the same Vaxine DC when running in the cluster. When running locally, you can use any string

Apply migrations locally

When running locally, you can apply migrations directly using make apply_migration. First make sure you’ve built your migrations in your application folder, then set the MIGRATIONS_DIR environment variable to the path to the migrations folder:

export MIGRATIONS_DIR='../path/to/migrations'

Now (re)run the electric service (with the env var set):

make shell

You can now apply named migrations using:

make apply_migration name=$MIGRATION_NAME

Where MIGRATION_NAME is the name of a migration folder created using electric migrations new, for example:

make apply_migration name=1666288253_create_items

Authentication

By default, in dev mode, electric uses insecure authentication. This just accepts a user id as the authentication token and authorizes the connection as that user.

Token based authentication requires a signed JWT token with a user_id claim, and a valid issuer.

To turn on token-based authentication in dev mode and when running in production, set the following environment variables:

  • SATELLITE_AUTH_SIGNING_KEY - Some random string used as the HMAC signing key. Must be at least 32 bytes long.

  • SATELLITE_AUTH_SIGNING_ISS - The JWT issuer (the iss field in the JWT).

You can generate a valid token using these configuration values by running mix electric.gen.token, e.g:

$ export SATELLITE_AUTH_SIGNING_KEY=00000000000000000000000000000000
$ export SATELLITE_AUTH_SIGNING_ISS=my.electric.server
$ mix electric.gen.token my_user my_other_user

The generated token(s) must be passed in the token field of the SatAuthReq protocol message.

For them to work, you must run the electric server configured with the same SATELLITE_AUTH_SIGNING_KEY and SATELLITE_AUTH_SIGNING_ISS set.

OSX networking

Note that if, when running on OSX, you get errors like:

could not connect to the publisher: connection to server at \"host.docker.internal\" (192.168.65.2), port 5433 failed

You may need to adjust your docker networking or run Electric within docker. To run within Docker, you can build the docker image locally:

make docker-build

And then run with the right env vars, e.g.:

docker run -it -p "5433:5433" -p "5133:5133" \
    -e "VAXINE_HOST=host.docker.internal"
    -e "ELECTRIC_HOST=host.docker.internal"
    -e "CONNECTORS=pg1=postgresql://electric:password@host.docker.internal:54321/electric;pg2=postgresql://electric:password@host.docker.internal:54322/electric" \
    docker.io/library/electric:local-build

Next steps

Continue with: