mirror of
https://github.com/netfun2000/hipudding-teslamate.git
synced 2026-02-27 09:44:28 +08:00
fdf8185ce2
* ci: add treefmt as code formatting multiplexer * style: linter findings in entrypoint.sh script * style: linter findings for yaml and yml * style: linter findings for json files * style: linter findings for nix files * style: linter findings for js files * style: linter findings for dashboards.sh * style: linter findings for md and mdx files * chore: remove unused clang formatter in treefmt config * style: linter findings for mdx files * ci: exclude Grafana dashboard JSON files from prettier formatting * Revert "style: linter findings for json files" This reverts commit f40c2e175992f66f887a3f4fc5aa1a1a89393147. * ci: exclude Grafana dashboard JSON files from all formatting as we use the grafana export style * style: linter findings for json files * doc: update changelog * ci(refactor): use composite action to avoid duplication in elixir workflow * doc: update changelog * ci: prevent workflow runs for certain conditions and allow scheduled runs * ci(refactor): use reusable workflow to check paths * ci(fix): correct output syntax for check_paths workflow and setting base branch * ci(refactor): use reusable workflows for streamlined DevOps pipeline * ci(fix): add write permission for packages in DevOps workflow * ci(test): test DevOps workflow * ci(test): test DevOps workflow * ci(fix): Update condition for spell_check, ensure_linting, elixir, and ghcr_build workflows to reflect empty result instead of false * ci: revert test DevOps * ci(refactor): allow ghcr_build parallel to elixir test * ci(refactor): Remove redundant check_paths job from elixir.yml, elixir_test.yml, and spell_check.yml workflows, check is done in devops.yml * feat: add treefmt-nix to nix flake (#4219 - @JakobLichterfeld) * ci: ensure proper linting via treefmt * ci(test): test ensure_linting workflow * ci(fix): checkout code for spell_checker to access file to check * ci(fix): allow impure in ensure_linting workflow * Revert "ci(test): test ensure_linting workflow" This reverts commit a67b17ec098a628a8b093ec20b34b0e2696811cc. * ci(fix): correct use of flake-utils for formatter and checks Co-authored-by: scottbot95 <scottbot95@gmail.com> * ci(fix): correct use of flake-utils for treefmt Co-authored-by: scottbot95 <scottbot95@gmail.com> * refactor: Remove unnecessary imports in flake * ci(fix): correct syntax in flake * ci(refactor): Remove unused code in flake.nix * style: standardised style for input url in flake * ci(fix): treefmt-nix config with existing options * ci(feat): Add Nix binary cache and update treefmt command in CI workflow * ci(refactor): Remove unused code in flake.nix * fix: include devShell packages only on supported platforms * fix: update hash for mix-deps package in flake.nix * ci(fix): Update treefmt command in CI workflow * ci(test): test ensure_linting workflow * feat: ensure mix deps are present in devShell * ci(feat): use flake-parts to enable treefmt-nix * feat: use flake-parts * fix: correct use of flake-parts for package build * doc: update CI badge URL for devops workflow * ci(fix): handle empty path filter output * ci: remove --impure flag from treefmt command in CI workflow * fix: correct treefmt.config settings in formatter.nix * fix: correct flake-parts inputs, avoid with in imports * fix: correct program name for mix-format in formatter.nix * feat: devenv via flake-parts * fix: correct use of legacy nix code with flake-parts * ci(fix): correct nix develop command in ensure_linting.yml * refactor: list imports explicitly in flake, rename folder to flake-modules to be precise * style: use tabs for indent size to format sh * style: use nixfmt-rfc-style * Revert "style: use nixfmt-rfc-style" This reverts commit 082056159e611c2c0639f8b4dc509446f5308a92. * style: use nixfmt style * fix: remove glibcLocales from optional dependencies to avoid "A definition for option `packages."[definition 4-entry 16]"' is not of type `package'." * fix: remove inotify-tools from optional dependencies to avoid "A definition for option `packages."[definition 4-entry 16]"' is not of type `package'." * fix: Remove inotify-tools and glibcLocales from optional dependencie * fix: correct file paths in flake.nix to version * fix: add ELIXIR_ERL_OPTIONS to shell environment to force utf8 locale * fix: add LOCALE_ARCHIVE to shell environment in flake.nix * Revert "fix: add LOCALE_ARCHIVE to shell environment in flake.nix" This reverts commit d45f6e32eea1fcaf55cd03733e648cda4dbb764b. * ci(refactor): rename workflow to elixir_dep_verification_and_static_analysis.yml to better reflect the intention,, remove duplicate checks * ci(debug): debug locale settings * Revert "ci(debug): debug locale settings" This reverts commit 9b402f3c5f9b41ff4716dc4fc99f711fc9ef8135. * Revert "fix: add ELIXIR_ERL_OPTIONS to shell environment to force utf8 locale" This reverts commit d02419cba44055d55978ad36d23493d8790ce21b. * fix: add LOCALE_ARCHIVE to shell environment in flake.nix * Revert "fix: add LOCALE_ARCHIVE to shell environment in flake.nix" This reverts commit 761b437e699d8381a0726dd969052b1e85c3f08f. * fix: add LANG=C.UTF-8 to shell environment in flake.nix * fix: add mix local.rebar and mix local.hex commands to flake.nix * fix: pin devenv to version without unix socket bug * chore: update nixpkgs to nixos-24.05 and update dependencies * doc: add treefmt config comments * ci: do not expose treefmt formatter programs in devshell * fix: correct use of module option to enable PostgreSQL server in flake.nix * Revert "chore: update nixpkgs to nixos-24.05 and update dependencies" This reverts commit a6ea3f25aa393c32d9c2494b21e6330de2482496. * feat: consistent use of erlang 26 and elixir 1_16 in flake * ci: switch to macOS runner for linting workflow * Revert "ci: do not expose treefmt formatter programs in devshell" This reverts commit 1ecfa451dcf4199554a5d96d241ee846601862d9. * Revert "ci: switch to macOS runner for linting workflow" This reverts commit 7b430667d8737e5adf91f5ba59bbf83eb15bcb7a. * ci: Remove nixpkgs channel specification in ensure_linting workflow * ci(debug): Add debug output for PATH and NIX_PATH in flake.nix * Revert "ci(debug): Add debug output for PATH and NIX_PATH in flake.nix" This reverts commit 07faec5ddf88195969f89f3e74791d030c980b03. * fix: avoid the need for impure for devenv see #4245 * fix: remove invalid custom build.check for formatter and use default * style: linter findings * fix: Add emptyTest to avoid nix flake check test execution on non-Linux systems * chore: Remove LANG=C.UTF-8 from enterShell in flake.nix * ci(fix): Remove --impure flag from treefmt command in CI mode * ci(fix): avoid impure mode in ensure_linting workflow * style: linter findings * ci(debug): debug elixir version and locale * chore: Update flake.lock dependencies * feat: use newer devenv as unix socket bug is fixed in upstream https://github.com/cachix/devenv/issues/1497 * fix: set rebar3 path in devenv * Revert "ci(debug): debug elixir version and locale" This reverts commit 7ecdc77c163cd6ef9c913689c1f75e86ce8587cb. * ci: re-enable path check in DevOps workflow * doc: update Development and Contributing guide with nix and treefmt * ci: use PostgreSQL 17 * style: linter findings * ci(fix): ensure cache name in build action does not contain invalid characters to avoid invalid reference format * doc: update changelog * Revert "ci(fix): ensure cache name in build action does not contain invalid characters to avoid invalid reference format" This reverts commit 02abb0359db4f22e5a28283476b3ced0d99729ff. * ci: remove branch restriction for check_paths workflow to increase sec * ci(fix): run ghcr build workflow only for specific conditions - Ensure workflow only run if there are no changes to the .github folder - Allow workflow to run on workflow call or PRs from forks - Prevent duplicate runs for PRs from non-forks - Avoid invalid reference format for cache name in PRs from our repository * doc: update changelog * fix: update hash for mix-deps package in flake.nix * fix: disable flakeCheck for formatter, as mix format need the dep to be fetched beforehand * ci(fix): run ghcr build workflow only for specific conditions * fix: move nixosModules.default to top-level attribute set * refactor: remove unnecessary config nesting in formatter.nix * ci(fix): ensure version for buildx is set to correct name --------- Co-authored-by: scottbot95 <scottbot95@gmail.com>
70 lines
4.8 KiB
Markdown
70 lines
4.8 KiB
Markdown
---
|
|
title: Import from tesla-apiscraper (BETA)
|
|
sidebar_label: tesla-apiscraper
|
|
---
|
|
|
|
This is a multi-step process to export your data from a [tesla-apiscraper](https://github.com/lephisto/tesla-apiscraper) InfluxDB backend, convert it to a format that can be imported into TeslaMate (specifically TeslaFi CSV) while fixing up some typical data glitches the scraper produces, and then import it.
|
|
|
|
## Requirements
|
|
|
|
- A copy of the data stored by `tesla-apiscraper` - specifically the `/opt/apiscraper/influxdb` folder mounted in the standard apiscraper Docker configuration. **Create a backup of this** before attempting the export and conversion, just in case.
|
|
|
|
- A system running Docker with sufficient RAM for InfluxDB to perform the CSV export. This does **not** need to be the same machine where API Scraper and/or TeslaMate was/is running - you can do the export and conversion on your PC/Mac. The important thing is to give the Docker machine more than 2GB of RAM, otherwise the InfluxDB export may fail.
|
|
|
|
- **CREATE A [BACKUP](../maintenance/backup_restore.md) OF YOUR DATA** before attempting to import anything into TeslaMate. This is all highly experimental and has only been successfully done once :)
|
|
|
|
## Instructions
|
|
|
|
:::note Troubleshooting
|
|
All of this is experimental and has not been extensively tested. If you encounter an error during Part 1 or Part 2, please create an issue in the [tesla-apiscraper-to-teslafi-export](https://github.com/olexs/tesla-apiscraper-to-teslafi-export) project on GitHub, since it's not directly related to TeslaMate.
|
|
:::
|
|
|
|
### Part 1: Exporting the API Scraper InfluxDB data into CSV
|
|
|
|
1. Download or clone the [tesla-apiscraper-to-teslafi-export](https://github.com/olexs/tesla-apiscraper-to-teslafi-export) repository into a folder on your machine.
|
|
|
|
2. Place the `tesla-apiscraper` data folder _contents_ (_data_, _meta_ and _wal_ folders) into the `influxdb-data` folder, next to the `influxdb-export.sh` file. The folder structure must look like this:
|
|
|
|
```console
|
|
.
|
|
├── influxdb-export.sh
|
|
├── influxdb-export.bat
|
|
├── (other stuff)
|
|
└── influxdb-data
|
|
├── README.md
|
|
├── data
|
|
├── meta
|
|
└── wal
|
|
```
|
|
|
|
3. Run the `influxdb-export.sh` script (or `influxdb-export.bat` if you're on Windows). You may need to `sudo` it / run it in an Administrator command line prompt if your Docker install needs root. It will do the following:
|
|
|
|
- Create the `influxdb-csv` output folder, if it doesn't exist yet
|
|
- Start an InfluxDB Docker container with the `influxdb-data` and `influxdb-csv` folders mounted (on Windows, you may need to allow your Docker to access the drive you are working on for the mounts to work)
|
|
- Wait for the container to report as _healthy_
|
|
- Execute the export commands for all data stored by apiscraper, putting the CSV files inside the `influxdb-csv` folder
|
|
- Stop and delete the InfluxDB container
|
|
|
|
This may take a little while. After the process has finished, if there are no errors reported, continue with the next part of the instructions.
|
|
|
|
### Part 2: Converting exported InfluxDB CSV files to TeslaFi CSV
|
|
|
|
1. Obtain your Tesla's **vehicle ID** number. It's a 10- or 11-digit number that uniquely identifies your car, and is part of the TeslaFi data format, but it's not included in tesla-apiscraper data - so you need to source it separately. There are several ways you can get it:
|
|
|
|
- Manually using the Tesla API. The number is listed under `vehicle_id` in the `vehicles` response, as documented here: [https://tesla-api.timdorr.com/api-basics/vehicles](https://tesla-api.timdorr.com/api-basics/vehicles).
|
|
- From the database of another Tesla API tracker you're already using, such as TeslaMate (**docker compose exec database psql teslamate teslamate -c 'select vid from cars;'**).
|
|
|
|
2. Run the `teslafi-convert.sh` script (or `teslafi-convert.bat` if you're on Windows). You may need to `sudo` it / run it in an Administrator command line prompt if your Docker install needs root. It will do the following:
|
|
|
|
- Create the `teslafi-csv` output folder, if it doesn't exist yet
|
|
- Build and start a Docker container with the `converter` app and its few dependencies, with the `influxdb-csv` and `teslafi-csv` folders mounted (on Windows, you may need to allow your Docker to access the drive you are working on for the mounts to work)
|
|
- Ask you for the vehicle ID mentioned above
|
|
- Process the CSV files in the `influxdb-csv` folder. This may take a couple minutes. Progress is displayed as the converter works its way through the files
|
|
- Stop and delete the Docker container
|
|
|
|
The finished TeslaFi-compatible CSV files are now located in the `teslafi-csv` folder.
|
|
|
|
### Part 3: Importing the processed CSV data into TeslaMate
|
|
|
|
- Proceed with the [TeslaFi import](teslafi.md) steps using the CSV files you just created.
|