Running the Examples
ElectricSQL provides a number of example applications on GitHub at github.com/electric-sql/examples. Some of which you can also see on our demos page.
These examples are configured to connect (when they’re online) to a sync service. By default, they use an existing cloud sync service, so you don’t have to run or provision anything. However, you have the option of setting up and using your own cloud sync service and/or running the backend stack yourself using our open source code.
This page walks through the options, so you can run the examples however you prefer.
Option 1 — use the default sync service
The simplest way is just to run the example application using the pre-configured sync service. See the instructions in the README.md
but you can usually run something like:
yarn
yarn start
This will run the app locally, connecting to an existing cloud sync service that’s already been configured with the right data model and any seed data.
Option 2 — connect to your own sync service
Sign up to ElectricSQL and follow the instructions to create a new app. After the system provisions the infrastructure, you’ll be able to view the connection details for your sync service and Postgres database. Copy these somewhere safe (like a password manager).
Install the CLI. For example using Homebrew, or see the install guide for more options:
brew install electric-sql/tap/electric
Login on the command line (using the email and password you signed up with):
electric auth login EMAIL
Make sure you’re in the root of your example folder and run (replacing APP
with the app
identifier from your sync service connection details you copied earlier):
electric config update --app APP
This updates the ./electric.json
config file and your migrations bundle to use the new app
identifier.
You can now write migrations to change your schema, for example:
electric migrations new "drop items and create kv"
This generates a subfolder at ./migrations/*_drop_items_and_create_kv/
containing a migration.sql
file. Open this file in your text editor and write the SQL DDL statements you want to create and alter your database schema. For example:
DROP TABLE IF EXISTS items;
CREATE TABLE IF NOT EXISTS kv (
key TEXT PRIMARY KEY NOT NULL
value TEXT
) WITHOUT ROWID;
Build your migrations:
electric build
Upload to your new cloud service:
electric sync
If you now run your app, it will apply the migrations to use your database schema and load the configuration in electric.json
to connect to your cloud sync service.
Option 3 — run the backend locally
So far we’ve been using the hosted cloud sync service. However, ElectricSQL is open source software and you can also run the backend database and sync service components yourself.
Make sure you have make
, Docker and at least Elixir 1.14 compiled with Erlang 24 installed.
Clone the electric-sql/electric repo:
git clone https://github.com/electric-sql/electric
cd electric
Fetch deps and compile:
make deps compile
Run the dependencies using:
make start_dev_env
Run the tests:
make tests
Develop using:
make shell
This runs active-active replication with Postgres over logical replication and exposes a protocol buffers API over web sockets on localhost:5133
.
For example to write some data into one of the Postgres instances:
docker exec -it -e PGPASSWORD=password electric_db_a_1 psql -h 127.0.0.1 -U electric -d electric
There’s also a second instance, electric-db_b_1
, if you want to see data being replicated between them.
Note that you can tear down all the containers with:
make stop_dev_env
Configure your application
Back in the root of your application repo, use the CLI to set the replication
option in your electric.json
:
electric config update --replication-disable-ssl --replication-host localhost --replication-port 5133
If you now run your application, you can change data using the GUI and monitor your terminal output to watch data flowing through the backend services. Also you can write data into Postgres and see it sync into the app.
Running the release or docker container
The Electric application is configured using environment variables. Everything that doesn’t have a default is required to run.
Variable | Default | Description |
---|---|---|
VAXINE_HOST |
Host of Vaxine instance to connect to | |
VAXINE_API_PORT |
8087 |
Port for the regular DB API on Vaxine instance |
VAXINE_REPLICATION_PORT |
8088 |
Port for the replication API on Vaxine instance |
VAXINE_CONNECTION_TIMEOUT |
5000 |
(ms) Timeout waiting while connecting to a Vaxine instance |
ELECTRIC_HOST |
Host of this electric instance for the reverse connection from Postgres. It has to be accessible from postgres instances listed in the CONNECTORS |
|
CONNECTORS |
"" |
Semicolon-separated list of Postgres connection strings for PG instances that will be part of the cluster |
POSTGRES_REPLICATION_PORT |
5433 |
Port for connections from PG instances as replication followers |
STATUS_PORT |
5050 |
Port to expose health and status API endpoint |
WEBSOCKET_PORT |
5133 |
Port to expose the /ws path for the replication over the websocket |
OFFSET_STORAGE_FILE |
./offset_storage_data.dat |
Path to the file storing the mapping between connected instances and offsets in Vaxine WAL. Should be persisted between Electric restarts. |
MIGRATIONS_DIR |
Directory to read the migration SQL files from (see below) | |
MIGRATIONS_FILE_NAME_SUFFIX |
/postgres.sql |
Suffix that is appended to the migration name when looking for the migration file |
SATELLITE_AUTH_SIGNING_KEY |
"" |
Authentication token signing/validation secret key. See below. |
SATELLITE_AUTH_SIGNING_ISS |
"" |
Cluster ID which acts as the issuer for the authentication JWT. See below. |
GLOBAL_CLUSTER_ID |
Identifier of the cluster within the Electric cloud. When running locally, you can use any string | |
ELECTRIC_INSTANCE_ID |
Unique identifier of this Electric instance when running in the cluster. When running locally, you can use any string | |
ELECTRIC_REGIONAL_ID |
Shared identifier for multiple Electric instance that connect to the same Vaxine DC when running in the cluster. When running locally, you can use any string |
Apply migrations locally
When running locally, you can apply migrations directly using make apply_migration
. First make sure you’ve built your migrations in your application folder, then set the MIGRATIONS_DIR
environment variable to the path to the migrations folder:
export MIGRATIONS_DIR='../path/to/migrations'
Now (re)run the electric service (with the env var set):
make shell
You can now apply named migrations using:
make apply_migration name=$MIGRATION_NAME
Where MIGRATION_NAME
is the name of a migration folder created using electric migrations new
, for example:
make apply_migration name=1666288253_create_items
Authentication
By default, in dev mode, electric uses insecure authentication. This just accepts a user id as the authentication token and authorizes the connection as that user.
Token based authentication requires a signed JWT token with a user_id
claim, and a valid issuer.
To turn on token-based authentication in dev mode and when running in production, set the following environment variables:
-
SATELLITE_AUTH_SIGNING_KEY
- Some random string used as the HMAC signing key. Must be at least 32 bytes long. -
SATELLITE_AUTH_SIGNING_ISS
- The JWT issuer (theiss
field in the JWT).
You can generate a valid token using these configuration values by running mix electric.gen.token
, e.g:
$ export SATELLITE_AUTH_SIGNING_KEY=00000000000000000000000000000000
$ export SATELLITE_AUTH_SIGNING_ISS=my.electric.server
$ mix electric.gen.token my_user my_other_user
The generated token(s) must be passed in the token
field of the SatAuthReq
protocol message.
For them to work, you must run the electric server configured with the same
SATELLITE_AUTH_SIGNING_KEY
and SATELLITE_AUTH_SIGNING_ISS
set.
OSX networking
Note that if, when running on OSX, you get errors like:
could not connect to the publisher: connection to server at \"host.docker.internal\" (192.168.65.2), port 5433 failed
You may need to adjust your docker networking or run Electric within docker. To run within Docker, you can build the docker image locally:
make docker-build
And then run with the right env vars, e.g.:
docker run -it -p "5433:5433" -p "5133:5133" \
-e "VAXINE_HOST=host.docker.internal"
-e "ELECTRIC_HOST=host.docker.internal"
-e "CONNECTORS=pg1=postgresql://electric:password@host.docker.internal:54321/electric;pg2=postgresql://electric:password@host.docker.internal:54322/electric" \
docker.io/library/electric:local-build
Next steps
Continue with:
- quickstart guide
- example applications on GitHub