archived-teslamate/website/docs/development.mdx

---
id: development
title: Development and Contributing
sidebar_label: Development and Contributing
---

## Requirements

- **Elixir** >= 1.18.4-otp-26
- **Postgres** >= 17.3
- An **MQTT broker** e.g. mosquitto (_optional_)
- **NodeJS** >= 20.15.0

or [Nix](https://nixos.org/download/). You can then use the nix devenv (via direnv) setup.

## Initial Setup

To run the TeslaMate test suite you need a database named `teslamate_test`:

```bash
# download dependencies, create the dev database and run migrations
mix setup

# create the test database
MIX_ENV=test mix ecto.setup
```

## Running locally

Start an iex session in another terminal window:

```elixir
iex -S mix phx.server
```

Then sign in with a Tesla account.

## Hot reloading

To immediately apply your local changes open or reload [http://localhost:4000](http://localhost:4000). You can also reload specific modules via `iex`, for example:

```elixir
iex> r TeslaMate.Vehicles.Vehicle
```

To only compile the changes:

```bash
mix compile
```

## Code formatting

### Format all files

Install [Treefmt](https://github.com/numtide/treefmt/releases) or use nix develop or the nix devenv (via direnv) setup.

```bash
treefmt
```

or with nix, when not using direnv:

```bash
nix develop --override-input devenv-root "file+file://"<(printf %s "$PWD") . --command treefmt
```

You can even use a VS Code extension like [treefmt](https://marketplace.visualstudio.com/items?itemName=ibecker.treefmt-vscode) to format the files on save.

### Only format elixir files

```bash
mix format
```

## Update pot files (extract messages for translation)

```bash
mix gettext.extract --merge
```

## Testing

To ensure a commit passes CI you should run `mix ci` locally, which executes the following commands:

- Check formatting (`mix format --check-formatted`)
- Run all tests (`mix test`)

### Testing with our CI which builds the Docker images automatically per PR

Our CI automatically builds the Docker images for each PR (only for repo own PRs, not for external PRs). To test the changes introduce by a PR you can edit your docker-compose.yml file as follows (replace `pr-3836` with the PR number):

For TeslaMate:

```yml
teslamate:
  # image: teslamate/teslamate:latest
  image: ghcr.io/teslamate-org/teslamate:pr-3836
```

For Grafana:

```yml
grafana:
  # image: teslamate/grafana:latest
  image: ghcr.io/teslamate-org/teslamate/grafana:pr-3836
```

## Making Changes to Grafana Dashboards

To update dashboards you need Grafana running locally. The following _docker-compose.yml_ can be used for this purpose:

```yml
services:
  grafana:
    image: teslamate/grafana:latest
    environment:
      - DATABASE_USER=postgres
      - DATABASE_PASS=postgres
      - DATABASE_NAME=teslamate_dev
      - DATABASE_HOST=host.docker.internal
    ports:
      - 3000:3000
    volumes:
      - grafana-data:/var/lib/grafana

volumes:
  grafana-data:
```

_(on Linux use the actual IP address of the host as `DATABASE_HOST`instead of `host.docker.internal`)_

Then build the image with `make grafana` and run the container via `docker compose up grafana`.

Access the Grafana at [http://localhost:3000](http://localhost:3000) and sign in with the default user `admin` and password `admin`.

Then edit the respective dashboard(s) locally. To export a dashboard hit the 'Save' button and select `Save JSON to file`. The final JSON file belongs in the directory `./grafana/dashboards/`. To apply the changes rebuild the image and start the container.

When making larger changes to Grafana Dashboards please include updated screenshots (1920x1080 @ x2, Grafana running in Kiosk mode) in `./website/static/screenshots`. Ensure to blur sensitive information.

## Best Practices

### Queries involving timestamp columns

Datetime values are currently stored in columns of type `timestamp`. [This is NOT recommended](https://wiki.postgresql.org/wiki/Don't_Do_This#Don.27t_use_timestamp_.28without_time_zone.29_to_store_UTC_times).

While [Grafana macros](https://grafana.com/docs/grafana/latest/datasources/postgres/#macros) like `$__timeFilter` & `$__timeGroup` are working PostgreSQL functions like `DATE_TRUNC()` require additional treatment.

Grafana is not setting the PostgreSQL session timezone. To ensure truncation is done with respect to the Grafana timezone setting, set the [optional time_zone argument](https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC) for `DATE_TRUNC()`.

```sql
DATE_TRUNC('day', TIMEZONE('UTC', date), '$__timezone')
```

In addition, ensure to compare either values with or without time zone.

### Streaming API data / positions table usage in dashboard queries

When Streaming API is enabled roughly 1 GB of data is gathered per car and 30 000 km. Most of that data (95+ percent) is stored in positions table. For optimal dashboard performance these recommendations should be followed:

- only query positions table when really needed
- if data in 15 second intervals is sufficient consider excluding streaming data by adding `ideal_battery_range_km IS NOT NULL and car_id = $car_id` as WHERE conditions

Before opening pull requests please diagnose index usage & query performance by making use of `EXPLAIN ANALYZE`.

### Enable _pg_stat_statements_ to collect query statistics

To quickly identify performance bottlenecks we encourage all contributors to enable the pg_stat_statements extension in their instance. For docker based installs you can follow these steps:

- Enable the pg_stat_statements module

  ```yml
  services:
    database:
      image: postgres:18-trixie
      ...
      command: postgres -c shared_preload_libraries=pg_stat_statements
      ...
  ```

- Create Extension to enable `pg_stat_statements` view

  ```sql
  CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
  ```

- Identify potentially slow queries (mean_exec_time)

  ```sql
  SELECT query, calls, mean_exec_time, total_exec_time FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10;
  ```

- Identify frequently executed queries (calls)

  ```sql
  SELECT query, calls, mean_exec_time, total_exec_time FROM pg_stat_statements ORDER BY calls DESC LIMIT 10;
  ```

Additional details about pg_stat_statements can be found [in the PostgreSQL docs](https://www.postgresql.org/docs/current/pgstatstatements.html)

## Entity Relationship Model (ERM)

import useBaseUrl from "@docusaurus/useBaseUrl";

<img
  alt="Entity Relationship Model"
  src={useBaseUrl("img/entity_relationship_model.png")}
/>

## Pre-Merge Checks for Dependency Updates

When reviewing a pull request that updates dependencies, it's crucial to verify that the changes are correct and don't break the Nix-based development environment. After checking out the branch of the PR, run the following commands:

- `nix flake check ...`: Verifies the flake's integrity across different systems.
- `nix build ...`: Ensures the project builds successfully with the new dependencies.
- `nix develop ... --command treefmt`: Checks if the code formatting still runs.
- `devenv up`: Confirms that the development environment starts up as expected.

```bash
# Run these commands to ensure everything works as expected
nix flake check --override-input devenv-root "file+file://"<(printf %s "$PWD") . --all-systems
nix build --override-input devenv-root "file+file://"<(printf %s "$PWD")
nix develop --override-input devenv-root "file+file://"<(printf %s "$PWD") . --command treefmt
devenv up
```

It is most likely you need to change the hash in `./nix/flake-modules/package.nix`. This could be `mixFodDeps`, `npmDepsHash`, or the `sha256` for `cldr`.

:::note
A `cldr` version update is particularly critical. The old hashes will still appear to work if you previously built using the old hash. But will silently get the old file not the new file.
:::