Compare commits

...

15 Commits

Author SHA1 Message Date
Arunavo Ray 91de0d1030 chore: bump version to 3.19.1 2026-06-14 12:07:48 +05:30
Brendan Davidson 85bd1f4042 Repository table bulk actions (#322)
* Handle indexing when shift + clicking in the repository table

* Move the buttons when selecting rows

* Add in a bulk delete func in the repositories table

* Add bulk delete handler

* Make the single action use the bulk delete

* Delete the single repository id handler
2026-06-14 10:14:51 +05:30
Brendan Davidson 4a28015685 Skip the user defined orgs to ignore (#323) 2026-06-14 10:14:48 +05:30
Arunavo Ray da23941369 chore: bump version to 3.19.0 2026-06-13 09:14:33 +05:30
Brendan Davidson 906ce57e8c Handle indexing when shift + clicking in the repository table (#316) 2026-06-13 09:14:02 +05:30
Arunavo Ray 1b84c75a97 chore: bump version to 3.18.0 2026-06-13 08:17:20 +05:30
ARUNAVO RAY 0b6b6b76bf feat(github): add skipPersonalRepos toggle to mirror only org repos (#304) (#320)
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
2026-06-13 08:00:50 +05:30
ARUNAVO RAY 7610a614da fix: scheduler auto-start gate, backup clone URL, cancel-pending action, actionable 405 (#319)
* fix(scheduler): make enabled flag authoritative for auto-start

checkAutoStartConfiguration() and performInitialAutoStart() previously
used `scheduleEnabled || hasMirrorInterval`, allowing a configured
GITEA_MIRROR_INTERVAL to trigger boot-time auto-start even after the
user disabled scheduling via the UI toggle.

env-config-loader already writes scheduleConfig.enabled=true when
GITEA_MIRROR_INTERVAL is set at container startup, so the interval is
a timing detail, not an enable signal. The documented env-var contract
is preserved: GITEA_MIRROR_INTERVAL at boot → env-config-loader sets
enabled=true → auto-start fires. But a later UI disable now sticks.

Add a focused unit test for the gate logic.

* fix(backup): always derive clone URL from user-configured Gitea URL

The pre-sync backup preferred repoInfo.clone_url, which reflects
Gitea's ROOT_URL setting. In Tailscale MagicDNS deployments (and any
setup where ROOT_URL is an external address), this URL is unreachable
from the app itself, causing bundle backup to fail.

Always build the clone URL as:
  ${config.giteaConfig.url.trimEnd('/')}/${owner}/${repo}.git

This matches the URL the app already uses for all other Gitea API
calls and is guaranteed reachable.

* feat(jobs): cancel-pending endpoint + fix misleading Delete All copy

Add POST /api/job/cancel-pending that sets the current user's
repositories with status "imported" or "failed" to "ignored",
preventing the scheduler from re-queuing them. In-flight "mirroring"
rows are left alone. Returns the count and logs one activity entry.

Fix the "Delete All Activities" dialog to clearly state it only clears
the history log and does not stop pending work. Rename button/title to
"Clear History" so intent is unambiguous.

Add a "Stop Pending Mirrors" button (StopCircle icon, amber) in both
mobile and desktop activity log toolbars, with a confirmation dialog
explaining repos are set to Ignored and can be re-enabled from the
Repositories page.

* fix(sync): actionable 405 error for non-pull-mirror repos

Gitea returns HTTP 405 with an empty body when the target repository is
no longer a pull mirror — e.g. the mirror was auto-disabled by Gitea or
the repository lost its mirror state after a manual edit.

Previously this fell through to the generic error handler which stored
the raw HttpError message (often empty) giving the user no guidance.

Now a 405 response is caught alongside the existing 400 handler and
sets the repository to "failed" with an actionable error message:

  "Gitea reports this repository is not a pull mirror (HTTP 405).
  In Gitea check Settings → Mirror Settings; if the mirror section is
  missing, delete the repository in Gitea and re-mirror it from
  gitea-mirror."

The same message is written to the activity log for visibility in the
dashboard.
2026-06-13 08:00:47 +05:30
ARUNAVO RAY c28dcc209f fix(releases): stop delete/recreate cycle on permanent order mismatch (#310) (#318)
Root cause (Theory A): the `needsRecreation` check compared GitHub
published_at-based expected indices against Gitea's API order. Gitea mirror
repos sort releases by tag-commit date, which can permanently disagree with
published_at order (e.g. unaconfig_dart v0.1.0 published after v0.1.1 but
tagged before). This made `currentExpectedIdx < nextExpectedIdx` evaluate
true on every sync, triggering delete-all-and-recreate forever — spamming
Gitea's activity feed with "released X" events (#310).

Fix: replace the destructive order-check machinery with set-based
reconciliation via `classifyReleasesForReconciliation`. Releases are
created when missing in Gitea and skipped (or PATCH-updated if content
drifted) when already present. No deletions are ever triggered by ordering.
Retain the existing release-limit trimming (retention cleanup) unchanged.

Also removes the 1-second per-release delay that was only needed for the
creation-order dance, significantly speeding up initial mirrors.

Adds unit tests covering: normal ordered repos, the unaconfig_dart inversion
fixture, missing→create, present→skip, and edge cases.
2026-06-13 08:00:44 +05:30
ARUNAVO RAY 40ee3cbc44 fix(mirror): reuse existing same-source mirrors instead of creating suffixed duplicates (#315) (#317)
Starred (and other) repos duplicated on every re-mirror (starred/Repo,
Repo-owner, Repo-owner-1, ...) because the existence check only asked
"does a repo with this name exist?" and never "is the existing repo a
mirror of THIS same source?". The repo's own prior mirror counted as a
collision, so generateUniqueRepoName bumped to the next suffix each run,
repointing mirroredLocation at the newest copy. Under a single re-call,
3 concurrent/retried jobs each computed a DIFFERENT suffixed name, so the
location-based in-flight guard never matched and the race produced extra
copies.

Fix (source-identity aware):
- New shared helper src/lib/utils/mirror-source-match.ts:
  - normalizeCloneUrl / cloneUrlsMatch: credential-, .git-, slash- and
    host-case-insensitive clone URL comparison.
  - isMirrorOfSource: a Gitea repo is "ours" only if it is a mirror AND
    its original_url matches this repo's source.
  - findExistingMirror: resolves an existing same-source mirror via the
    recorded mirroredLocation first (survives strategy changes — #309),
    then the base candidate name.
  - classifyCandidateName: pure available/reusable/taken decision.
- gitea-enhanced: export GiteaRepoInfo and add original_url (Gitea's
  recorded migration source) for source matching.
- Both create paths (mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg):
  run findExistingMirror BEFORE name generation; on a hit, reuse that
  location and route into the existing "already mirrored" handling rather
  than calling generateUniqueRepoName. Names now converge under
  concurrency so the in-flight guard becomes effective.
- generateUniqueRepoName is now source-aware: an occupied name held by a
  mirror of the SAME source is reused (no suffix); suffixing only happens
  on a genuine different-source collision, preserving #95/#236 behavior.
  The per-user DB claim check is retained so two users mirroring the same
  source into a shared org stay separated.
- Phantom-fork guard (#309): the existingRepoInfo.mirror branches now
  verify same-source before marking "mirrored"; on mismatch they fall
  through to unique-name generation and create a separate mirror.
- Scheduler: a `failed` repo whose mirroredLocation still resolves to a
  live same-source mirror is routed to syncGiteaRepo instead of re-create,
  breaking the failed-metadata re-create loop cheaply.
- Remove dead src/lib/starred-repos-handler.ts (zero importers across all
  git history); its correct base-name/.mirror reuse logic now lives in the
  shared helper.

Tests: src/lib/utils/mirror-source-match.test.ts (30 cases) covers URL
normalization, reuse at base name, reuse via mirroredLocation across a
strategy change, genuine different-source collision (suffix), phantom
fork, stale mirroredLocation fallback, per-user DB-claim separation, and
the suffix-vs-reuse classification. Full suite: 319 pass, 0 fail.
2026-06-13 08:00:41 +05:30
Arunavo Ray 699a5771f5 chore: bump version to 3.17.1 2026-06-05 18:42:11 +05:30
ARUNAVO RAY e862714d6a fix(db): self-heal sso_providers duplicate-column crash on upgrade (#312) (#313)
Migration 0013 runs as a single transaction that rebuilds `organizations`
and then `ALTER TABLE sso_providers ADD saml_config` / `ADD domain_verified`.
On instances where those columns were already created outside Drizzle (declared
in schema.ts and added via db:push / an SSO-register round-trip on an
intermediate build), the ADD throws "duplicate column name: saml_config". That
rolls back the entire 0013 transaction, so 0013 is never recorded in
`__drizzle_migrations` and is retried — failing identically — on every boot,
crash-looping the server.

Add a pre-migrate repair (mirroring the existing repairFailedMigrations() for
the 0009 case): when 0013 is unrecorded but the columns already exist, preserve
any real SAML provider config, drop the stranded columns so the canonical 0013
runs in full (organizations rebuild included), then restore the preserved
values once the columns are re-added. No-op on fresh installs, clean upgrades,
and already-migrated databases.

This lets affected instances recover automatically on the next boot after
upgrading — no manual SQLite surgery required.

- src/lib/db/migration-repairs.ts: repairDuplicateSsoColumns + restoreSsoDataAfter0013
- src/lib/db/index.ts: wire both around migrate()
- scripts/validate-migrations.ts: cover the broken-upgrade + data-preservation path
2026-06-05 18:41:46 +05:30
Arunavo Ray 716981aa04 chore: bump version to 3.17.0 2026-06-02 11:41:24 +05:30
ARUNAVO RAY 66e3284898 fix(sso): repair SSO login bounce + migrate to @better-auth/oauth-provider (#307)
Resolves #306. SSO sign-in via OIDC (Authentik / Keycloak / etc.) now links the
SSO identity to an existing email/password admin instead of bouncing to /login
with `?error=UNKNOWN`. Account-linking is gated on the operator-supplied
**Domain** field — cross-domain claims from a compromised IdP are refused.

Also bundles the deprecated `oidcProvider` → `@better-auth/oauth-provider`
migration. **Operators using the OAuth-provider feature must rotate registered
client secrets after upgrade** (legacy plaintext → hashed storage; see the
0012 migration notes).

Verified end-to-end on the pr-307 image against a real Authentik instance:
SSO login lands on the dashboard, `accounts` table gets both `credential` and
`authentik` rows for the same user. See PR description for full details.
2026-06-02 11:40:54 +05:30
IYUANWEIZE 695f0ff005 Fix Docker image line in .env.example (#305) 2026-05-28 13:49:32 +05:30
45 changed files with 7502 additions and 892 deletions
+4 -1
View File
@@ -21,6 +21,9 @@ DATABASE_URL=sqlite://data/gitea-mirror.db
BETTER_AUTH_SECRET=change-this-to-a-secure-random-string-in-production
BETTER_AUTH_URL=http://localhost:4321
# ENCRYPTION_SECRET=optional-encryption-key-for-token-encryption # Generate with: openssl rand -base64 48
# Better Auth log verbosity: debug | info | warn | error (default: warn).
# Set to "debug" to trace SSO/OIDC sign-in and callback issues.
# BETTER_AUTH_LOG_LEVEL=debug
# ===========================================
# REVERSE PROXY CONFIGURATION
@@ -60,7 +63,7 @@ PUBLIC_BETTER_AUTH_URL=http://localhost:4321
# Docker Registry Configuration
DOCKER_REGISTRY=ghcr.io
DOCKER_IMAGE=raylabshq/gitea-mirror:
DOCKER_IMAGE=raylabshq/gitea-mirror
DOCKER_TAG=latest
# ===========================================
+18
View File
@@ -88,6 +88,24 @@ jobs:
- name: Start Gitea and git-server containers
run: |
# Retry pulls before `up` — Docker Hub occasionally returns
# "context deadline exceeded" on the first attempt. Pulling
# separately also keeps the failure mode obvious if it persists.
echo "Pulling images..."
for attempt in 1 2 3; do
if docker compose -f tests/e2e/docker-compose.e2e.yml pull --quiet; then
echo "✓ Images pulled (attempt $attempt)"
break
fi
if [ "$attempt" -eq 3 ]; then
echo "ERROR: docker compose pull failed after 3 attempts"
exit 1
fi
sleep_s=$((attempt * 10))
echo "Pull attempt $attempt failed, retrying in ${sleep_s}s..."
sleep "$sleep_s"
done
echo "Starting containers via docker compose..."
docker compose -f tests/e2e/docker-compose.e2e.yml up -d
+1
View File
@@ -52,3 +52,4 @@ tests/e2e/git-repos/
/blob-report/
/playwright/.cache/
/playwright/.auth/
.playwright-mcp/
+3
View File
@@ -9,6 +9,7 @@
"@astrojs/mdx": "5.0.0",
"@astrojs/node": "10.0.1",
"@astrojs/react": "^5.0.0",
"@better-auth/oauth-provider": "1.6.11",
"@better-auth/sso": "1.6.11",
"@octokit/plugin-throttling": "^11.0.3",
"@octokit/rest": "^22.0.1",
@@ -175,6 +176,8 @@
"@better-auth/mongo-adapter": ["@better-auth/mongo-adapter@1.6.11", "", { "peerDependencies": { "@better-auth/core": "^1.6.11", "@better-auth/utils": "0.4.0", "mongodb": "^6.0.0 || ^7.0.0" }, "optionalPeers": ["mongodb"] }, "sha512-3Tor8rSv8vSEIMEaV2PFpPEuVhqc1gNoZ6eGvoh3LwExXXuj8madew6ob+H1pH7Aphn3Ar5PQ08AguT8TbwFAA=="],
"@better-auth/oauth-provider": ["@better-auth/oauth-provider@1.6.11", "", { "dependencies": { "jose": "^6.1.3", "zod": "^4.3.6" }, "peerDependencies": { "@better-auth/core": "^1.6.11", "@better-auth/utils": "0.4.0", "@better-fetch/fetch": "1.1.21", "better-auth": "^1.6.11", "better-call": "1.3.5" } }, "sha512-iMywpOEAiAUdtvpaRS8yKye+wO3AieOB3Sfv8czkmPduzFuKBICCWuOEAElQEk5tQz3vzWx64zNlLBkgEAOhuw=="],
"@better-auth/prisma-adapter": ["@better-auth/prisma-adapter@1.6.11", "", { "peerDependencies": { "@better-auth/core": "^1.6.11", "@better-auth/utils": "0.4.0", "@prisma/client": "^5.0.0 || ^6.0.0 || ^7.0.0", "prisma": "^5.0.0 || ^6.0.0 || ^7.0.0" }, "optionalPeers": ["@prisma/client", "prisma"] }, "sha512-Pw+7q7zTp+VSci1V+CYMvuxIbAeVMZLe4lRo46LJoAKMHfjFl5T/ycsyFvWs/DkWC7n9gZZzRDEbHp0I5FiKKw=="],
"@better-auth/sso": ["@better-auth/sso@1.6.11", "", { "dependencies": { "fast-xml-parser": "^5.5.7", "jose": "^6.1.3", "samlify": "~2.10.2", "tldts": "^6.1.0", "zod": "^4.3.6" }, "peerDependencies": { "@better-auth/core": "^1.6.11", "@better-auth/utils": "0.4.0", "@better-fetch/fetch": "1.1.21", "better-auth": "^1.6.11", "better-call": "1.3.5" } }, "sha512-lJHmoCayp9Woh/MPKTHDfGq7k1oQbU2yz5tIOZXl/pzrgLxV7fMGo9aJCyabHkw3GHMjBes4byC6aakHYzpZIg=="],
+2 -1
View File
@@ -40,6 +40,7 @@ Essential application settings required for running Gitea Mirror.
| `BETTER_AUTH_URL` | Authentication origin (scheme + host only, e.g. `https://git.example.com`). Do **not** include a path — any path is automatically stripped, and `BASE_URL` is applied separately. | `http://localhost:4321` | No |
| `PUBLIC_BETTER_AUTH_URL` | Client-side auth origin for multi-origin access (same rule: origin only, no path). Set this to your primary domain when you need to access the app from different origins (e.g., both IP and domain). The client will use this URL for all auth requests instead of the current browser origin. | - | No |
| `BETTER_AUTH_TRUSTED_ORIGINS` | Trusted origins for authentication requests. Comma-separated list of URLs. Use this to specify additional access URLs (e.g., local IP + domain: `http://10.10.20.45:4321,https://gitea-mirror.mydomain.tld`), SSO providers, reverse proxies, etc. | - | No |
| `BETTER_AUTH_LOG_LEVEL` | Better Auth logger verbosity. Set to `debug` to surface the full SSO/OIDC sign-in and callback trace when troubleshooting authentication. Accepted values: `debug`, `info`, `warn`, `error`. (Better Auth does **not** use the `DEBUG` env var.) | `warn` | No |
| `ENCRYPTION_SECRET` | Optional encryption key for tokens (generate with: `openssl rand -base64 48`) | - | No |
## HTTPS / TLS
@@ -113,7 +114,7 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
|----------|-------------|---------|---------|
| `MIRROR_ORGANIZATIONS` | Mirror organization repositories | `false` | `true`, `false` |
| `PRESERVE_ORG_STRUCTURE` | Preserve GitHub organization structure in Gitea | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal) | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal); sets `skipPersonalRepos: true` in GitHub config | `false` | `true`, `false` |
| `MIRROR_STRATEGY` | Repository organization strategy | `preserve` | `preserve`, `single-org`, `flat-user`, `mixed` |
### Advanced Settings
+20 -3
View File
@@ -28,7 +28,7 @@ SSO allows your users to sign in using external identity providers like Google,
#### Required Fields
- **Issuer URL**: The OIDC issuer URL (e.g., `https://accounts.google.com`)
- **Domain**: The email domain for this provider (e.g., `example.com`)
- **Domain**: The email domain this provider serves (e.g., `example.com`). **This is load-bearing for account-linking trust — see [Account Linking](#account-linking) below.** Multi-domain IdPs can use a comma-separated list (`example.com,subsidiary.com`).
- **Provider ID**: A unique identifier for this provider (e.g., `google-sso`)
- **Client ID**: The OAuth client ID from your provider
- **Client Secret**: The OAuth client secret from your provider
@@ -57,6 +57,22 @@ https://your-domain.com/api/auth/sso/callback/{provider-id}
Replace `{provider-id}` with your chosen Provider ID.
### Account Linking
When a user signs in via SSO whose email matches an **existing** email/password account, Gitea Mirror will try to *link* the SSO identity to that account so the user lands on the same dashboard — instead of being bounced to the login page or creating a duplicate account.
The trust model is **domain-scoped**:
- Every SSO provider you register is automatically marked `domainVerified: true` for the **Domain** field you set on registration.
- An SSO sign-in is auto-linked to an existing local account **only when** the user's email address actually belongs to that registered domain. Cross-domain emails are rejected even if the provider is registered.
- Local email/password accounts in this app are never verified (no email-verification flow exists), so we deliberately turn off `requireLocalEmailVerified` on the linker — otherwise no one could ever link.
**What this means for you:**
- Set the **Domain** field to the email domain(s) your IdP actually issues identities for. Don't set it to `example.com` if your IdP issues `@elsewhere.com` emails — auto-linking will silently refuse them.
- If your IdP allows users to self-register or claim arbitrary emails *within the registered domain*, an attacker on that IdP can absorb a local account by claiming the same email. This is a trust decision: by registering an IdP, you're vouching for its identity model for that domain.
- Multi-domain IdPs (one Authentik serving `acme.com,acquired.io`) work fine — list every domain comma-separated in the **Domain** field.
### Example: Google SSO Setup
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
@@ -99,7 +115,7 @@ Working Authentik deployments (see [#134](https://github.com/RayLabsHQ/gitea-mir
Notes:
- Make sure `BETTER_AUTH_URL` and (if you serve the UI from multiple origins) `BETTER_AUTH_TRUSTED_ORIGINS` point at the public URL users reach. A mismatch can surface as 500 errors after redirect.
- Authentik must report the users email as verified (default behavior) so Gitea Mirror can auto-link accounts.
- Set the **Domain** field to the email domain your Authentik users actually have. Auto-linking to an existing local admin only happens for emails in that domain — see [Account Linking](#account-linking) for the trust model. (Authentik's default email scope mapping returns `email_verified: False` for OIDC clients, which is why account linking is gated on the domain match here rather than the IdP's verified-email claim.)
- If you created an Authentik provider before v3.8.10 you should delete it and re-add it after upgrading; older versions saved incomplete endpoint data which leads to the `url.startsWith` error explained in the Troubleshooting section.
## Setting up Header / Forward Authentication
@@ -257,7 +273,8 @@ When an application requests authentication:
1. **"Invalid origin" error**: Check that your Gitea Mirror URL matches the configured redirect URI
2. **"Provider not found" error**: Ensure the provider is properly configured and enabled
3. **Redirect loop**: Verify the redirect URI in both Gitea Mirror and the SSO provider match exactly
4. **`TypeError: undefined is not an object (evaluating 'url.startsWith')`**: This indicates the stored provider configuration is missing OIDC endpoints. Delete the provider from Gitea Mirror and re-register it using the **Discover** button so authorization/token URLs are saved (see [#73](https://github.com/RayLabsHQ/gitea-mirror/issues/73) and [#122](https://github.com/RayLabsHQ/gitea-mirror/issues/122) for examples).
4. **`?error=UNKNOWN` on the homepage after a successful upstream login** (or `?error=account%20not%20linked` in development): the SSO callback succeeded but Better Auth refused to link the SSO identity to an existing local account. The most common cause is the SSO provider's registered **Domain** not matching the user's actual email domain — see [Account Linking](#account-linking). Set `BETTER_AUTH_LOG_LEVEL=debug` to see Better Auth's full callback trace (look for "User already exist but account isn't linked to ...") and confirm the diagnosis. The same symptom in production gets sanitized to `UNKNOWN` by Better Auth's error page before the redirect, which is why the visible error is opaque.
5. **`TypeError: undefined is not an object (evaluating 'url.startsWith')`**: This indicates the stored provider configuration is missing OIDC endpoints. Delete the provider from Gitea Mirror and re-register it using the **Discover** button so authorization/token URLs are saved (see [#73](https://github.com/RayLabsHQ/gitea-mirror/issues/73) and [#122](https://github.com/RayLabsHQ/gitea-mirror/issues/122) for examples).
### OIDC Provider Issues
+50 -2
View File
@@ -125,11 +125,59 @@ npm start
### Debug Mode:
Enable debug logging by setting environment variable:
> **Note:** Better Auth uses its own logger and does **not** read the `DEBUG`
> environment variable (it is not based on the `debug` npm package). An older
> version of this guide suggested `DEBUG=better-auth:*` — that has no effect.
Better Auth's logger defaults to the `warn` level, so SSO/OIDC sign-in and
callback details are hidden. Set the log level to `debug` to surface the full
trace:
```bash
DEBUG=better-auth:* bun run dev
# Local dev
BETTER_AUTH_LOG_LEVEL=debug bun run dev
```
```yaml
# Docker Compose
services:
gitea-mirror:
environment:
- BETTER_AUTH_LOG_LEVEL=debug
```
Then watch the server logs (e.g. `docker compose logs -f gitea-mirror`) while you
attempt an SSO login. Lines are prefixed with `[Better Auth]:`. Accepted values
are `debug`, `info`, `warn`, and `error`.
#### Debugging a login that bounces back to `/login`
If clicking the SSO button sends you to the provider and then straight back to
the login screen, the OAuth flow itself usually succeeded but **no session
cookie was persisted**. Work through these checks:
1. **Enable `BETTER_AUTH_LOG_LEVEL=debug`** (above) and look for errors during
the `/api/auth/sso/callback/<provider-id>` request.
2. **Check for `?error=UNKNOWN` on the landing URL** (or `?error=account%20not%20linked` in dev).
That's Better Auth's account-linking step refusing to attach the SSO identity
to an existing email/password account. The debug log line to look for is
`User already exist but account isn't linked to <providerId>`. The fix is
almost always to set the SSO provider's **Domain** field to the email domain
your users actually have — auto-linking is gated on that domain match.
See [docs/SSO-OIDC-SETUP.md#account-linking](./SSO-OIDC-SETUP.md#account-linking).
3. **Check the redirect URI** registered in your IdP exactly matches
`https://<your-domain>/api/auth/sso/callback/<provider-id>` (scheme, host,
and provider ID — no trailing slash).
4. **Confirm the session cookie is set.** In the browser DevTools → Network,
inspect the callback response for a `Set-Cookie: better-auth-session=…`
header, and DevTools → Application → Cookies for the stored cookie. Behind a
reverse proxy, ensure `BETTER_AUTH_URL` is your **external HTTPS** URL so the
cookie is issued with the correct domain and `Secure` flag, and that the
proxy forwards `X-Forwarded-Proto: https` and `X-Forwarded-Host`.
5. **A `401` on `/api/sso/applications` is unrelated** to client login — that
endpoint backs the OAuth *provider* (consent) management UI and requires an
existing session. It is not part of the Authentik/OIDC sign-in flow.
## Testing Different Scenarios
### 1. New User Registration
+126
View File
@@ -0,0 +1,126 @@
-- Migrate the OAuth/OIDC *provider* feature from the deprecated
-- `oidc-provider` plugin to `@better-auth/oauth-provider`.
--
-- Tables: oauth_applications -> oauth_clients, oauth_access_tokens reshaped,
-- new oauth_refresh_tokens, oauth_consent -> oauth_consents, plus a `jwks`
-- table for the `jwt` plugin (id_token signing keys).
--
-- Data preservation:
-- * Registered clients are copied from oauth_applications into oauth_clients,
-- converting the legacy comma-separated `redirect_urls` into the JSON
-- string[] (`redirect_uris`) the new adapter expects.
-- * Access tokens and consent records are NOT migrated: access tokens are
-- short-lived (and the column shape changed entirely), and consents are
-- cheaply re-granted on next authorization. Both old tables are dropped.
--
-- NOTE: legacy client secrets were stored in plaintext, whereas the new
-- provider stores them hashed. Migrated client secrets will therefore not
-- validate as-is — affected applications must rotate their secret after
-- upgrade.
CREATE TABLE `oauth_clients` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`client_secret` text,
`name` text,
`disabled` integer DEFAULT false,
`skip_consent` integer,
`enable_end_session` integer,
`subject_type` text,
`scopes` text,
`user_id` text,
`uri` text,
`icon` text,
`contacts` text,
`tos` text,
`policy` text,
`software_id` text,
`software_version` text,
`software_statement` text,
`redirect_uris` text NOT NULL,
`post_logout_redirect_uris` text,
`token_endpoint_auth_method` text,
`grant_types` text,
`response_types` text,
`public` integer,
`type` text,
`require_pkce` integer,
`reference_id` text,
`metadata` text,
`created_at` integer DEFAULT (unixepoch()),
`updated_at` integer DEFAULT (unixepoch()),
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_clients_client_id_unique` ON `oauth_clients` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_clients_client_id` ON `oauth_clients` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_clients_user_id` ON `oauth_clients` (`user_id`);--> statement-breakpoint
INSERT INTO `oauth_clients` (
`id`, `client_id`, `client_secret`, `name`, `disabled`, `user_id`,
`redirect_uris`, `type`, `metadata`, `created_at`, `updated_at`
)
SELECT
`id`, `client_id`, `client_secret`, `name`, `disabled`, `user_id`,
'["' || replace(`redirect_urls`, ',', '","') || '"]',
`type`, `metadata`, `created_at`, `updated_at`
FROM `oauth_applications`;--> statement-breakpoint
DROP TABLE `oauth_applications`;--> statement-breakpoint
DROP TABLE `oauth_access_tokens`;--> statement-breakpoint
CREATE TABLE `oauth_access_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text,
`client_id` text NOT NULL,
`session_id` text,
`user_id` text,
`reference_id` text,
`refresh_id` text,
`expires_at` integer,
`created_at` integer DEFAULT (unixepoch()),
`scopes` text NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_access_tokens_token_unique` ON `oauth_access_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_token` ON `oauth_access_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_client_id` ON `oauth_access_tokens` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_user_id` ON `oauth_access_tokens` (`user_id`);--> statement-breakpoint
CREATE TABLE `oauth_refresh_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`client_id` text NOT NULL,
`session_id` text,
`user_id` text NOT NULL,
`reference_id` text,
`expires_at` integer,
`created_at` integer DEFAULT (unixepoch()),
`revoked` integer,
`auth_time` integer,
`scopes` text NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_refresh_tokens_token_unique` ON `oauth_refresh_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_token` ON `oauth_refresh_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_client_id` ON `oauth_refresh_tokens` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_user_id` ON `oauth_refresh_tokens` (`user_id`);--> statement-breakpoint
DROP TABLE `oauth_consent`;--> statement-breakpoint
CREATE TABLE `oauth_consents` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`user_id` text,
`reference_id` text,
`scopes` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()),
`updated_at` integer DEFAULT (unixepoch()),
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_consents_client_id` ON `oauth_consents` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consents_user_id` ON `oauth_consents` (`user_id`);--> statement-breakpoint
CREATE TABLE `jwks` (
`id` text PRIMARY KEY NOT NULL,
`public_key` text NOT NULL,
`private_key` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`expires_at` integer
);
+35
View File
@@ -0,0 +1,35 @@
PRAGMA foreign_keys=OFF;--> statement-breakpoint
CREATE TABLE `__new_organizations` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`normalized_name` text NOT NULL,
`avatar_url` text NOT NULL,
`membership_role` text DEFAULT 'member' NOT NULL,
`is_included` integer DEFAULT true NOT NULL,
`destination_org` text,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`repository_count` integer DEFAULT 0 NOT NULL,
`public_repository_count` integer,
`private_repository_count` integer,
`fork_repository_count` integer,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
INSERT INTO `__new_organizations`("id", "user_id", "config_id", "name", "normalized_name", "avatar_url", "membership_role", "is_included", "destination_org", "status", "last_mirrored", "error_message", "repository_count", "public_repository_count", "private_repository_count", "fork_repository_count", "created_at", "updated_at") SELECT "id", "user_id", "config_id", "name", "normalized_name", "avatar_url", "membership_role", "is_included", "destination_org", "status", "last_mirrored", "error_message", "repository_count", "public_repository_count", "private_repository_count", "fork_repository_count", "created_at", "updated_at" FROM `organizations`;--> statement-breakpoint
DROP TABLE `organizations`;--> statement-breakpoint
ALTER TABLE `__new_organizations` RENAME TO `organizations`;--> statement-breakpoint
PRAGMA foreign_keys=ON;--> statement-breakpoint
CREATE INDEX `idx_organizations_user_id` ON `organizations` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_config_id` ON `organizations` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_status` ON `organizations` (`status`);--> statement-breakpoint
CREATE INDEX `idx_organizations_is_included` ON `organizations` (`is_included`);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_organizations_user_normalized_name` ON `organizations` (`user_id`,`normalized_name`);--> statement-breakpoint
ALTER TABLE `sso_providers` ADD `saml_config` text;--> statement-breakpoint
ALTER TABLE `sso_providers` ADD `domain_verified` integer DEFAULT true NOT NULL;
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+15 -1
View File
@@ -85,6 +85,20 @@
"when": 1774058400000,
"tag": "0011_notification_config",
"breakpoints": true
},
{
"idx": 12,
"version": "6",
"when": 1774062000000,
"tag": "0012_oauth_provider_migration",
"breakpoints": true
},
{
"idx": 13,
"version": "6",
"when": 1780377747526,
"tag": "0013_slim_galactus",
"breakpoints": true
}
]
}
}
+2 -1
View File
@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.16.3",
"version": "3.19.1",
"engines": {
"bun": ">=1.2.9"
},
@@ -58,6 +58,7 @@
"@astrojs/mdx": "5.0.0",
"@astrojs/node": "10.0.1",
"@astrojs/react": "^5.0.0",
"@better-auth/oauth-provider": "1.6.11",
"@better-auth/sso": "1.6.11",
"@octokit/plugin-throttling": "^11.0.3",
"@octokit/rest": "^22.0.1",
+212 -1
View File
@@ -3,6 +3,10 @@
import { Database } from "bun:sqlite";
import { readFileSync } from "fs";
import path from "path";
import {
repairDuplicateSsoColumns,
restoreSsoDataAfter0013,
} from "../src/lib/db/migration-repairs";
type JournalEntry = {
idx: number;
@@ -188,6 +192,202 @@ function verify0011Migration(db: any) {
assert(parsed.provider === "ntfy", "Expected default notification_config.provider to be 'ntfy'");
}
function seedPre0012Database(db: any) {
// The harness has already run migrations 0000-0011, so the legacy
// oidc-provider tables exist. Seed a registered client (with the legacy
// comma-separated redirect_urls format) plus the related token/consent rows
// to exercise the create/transform/drop paths in 0012.
db.run("INSERT INTO users (id, email, username, name) VALUES ('u1', 'u1@example.com', 'user1', 'User One')");
db.run("INSERT INTO oauth_applications (id, client_id, client_secret, name, redirect_urls, type, disabled, user_id) VALUES ('app1', 'client-1', 'secret-1', 'Example App', 'https://example.com/callback,https://example.com/cb2', 'web', false, 'u1')");
db.run("INSERT INTO oauth_access_tokens (id, access_token, refresh_token, access_token_expires_at, refresh_token_expires_at, client_id, user_id, scopes) VALUES ('oat1', 'tok', 'rtok', 7000, 8000, 'client-1', 'u1', '[\"repo\"]')");
db.run("INSERT INTO oauth_consent (id, user_id, client_id, scopes, consent_given) VALUES ('cons1', 'u1', 'client-1', '[\"repo\"]', true)");
}
function verify0012Migration(db: any) {
// Old provider tables are dropped.
for (const table of ["oauth_applications", "oauth_consent"]) {
const row = db
.query("SELECT name FROM sqlite_master WHERE type='table' AND name = ?")
.get(table) as { name: string } | null;
assert(!row, `Expected ${table} table to be dropped after migration`);
}
// New provider tables exist.
for (const table of ["oauth_clients", "oauth_access_tokens", "oauth_refresh_tokens", "oauth_consents", "jwks"]) {
const row = db
.query("SELECT name FROM sqlite_master WHERE type='table' AND name = ?")
.get(table) as { name: string } | null;
assert(row, `Expected ${table} table to exist after migration`);
}
// The registered client is preserved and its redirect URIs converted from
// the legacy comma-separated string into a JSON string[].
const client = db
.query("SELECT client_id, client_secret, name, redirect_uris, type, user_id FROM oauth_clients WHERE id = 'app1'")
.get() as { client_id: string; client_secret: string; name: string; redirect_uris: string; type: string; user_id: string } | null;
assert(client, "Expected migrated oauth_clients row for app1");
assert(client.client_id === "client-1", "Expected client_id to be preserved");
assert(client.name === "Example App", "Expected client name to be preserved");
assert(client.user_id === "u1", "Expected owner user_id to be preserved");
const uris = JSON.parse(client.redirect_uris);
assert(
Array.isArray(uris) && uris.length === 2 && uris[0] === "https://example.com/callback" && uris[1] === "https://example.com/cb2",
`Expected redirect_uris to be a JSON array of the two callbacks, got ${client.redirect_uris}`,
);
// The reshaped tables accept the new column layout.
db.run("INSERT INTO oauth_clients (id, client_id, redirect_uris) VALUES ('app2', 'client-2', '[\"https://example.com/cb\"]')");
db.run("INSERT INTO oauth_refresh_tokens (id, token, client_id, user_id, scopes) VALUES ('rt1', 'refresh-1', 'client-2', 'u1', '[\"openid\"]')");
db.run("INSERT INTO oauth_access_tokens (id, token, client_id, user_id, scopes) VALUES ('at1', 'access-1', 'client-2', 'u1', '[\"openid\"]')");
db.run("INSERT INTO oauth_consents (id, client_id, user_id, scopes) VALUES ('co1', 'client-2', 'u1', '[\"openid\"]')");
db.run("INSERT INTO jwks (id, public_key, private_key) VALUES ('jwk1', 'public', 'private')");
}
function seedPre0013Database(db: any) {
// Migrations 0000-0012 have run, so sso_providers lacks samlConfig /
// domainVerified and the organizations table still carries the inherited
// DEFAULT '' on normalized_name from 0007. Seed both so the table-rebuild
// and the column-adds can be verified end-to-end.
db.run("INSERT INTO users (id, email, username, name) VALUES ('u-sso', 'sso@example.com', 'sso', 'SSO User')");
db.run("INSERT INTO configs (id, user_id, name, is_active, github_config, gitea_config, schedule_config, cleanup_config) VALUES ('cfg-pre13', 'u-sso', 'Default', 1, '{}', '{}', '{}', '{}')");
db.run("INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id) VALUES ('sso-pre13', 'https://idp.example.com', 'example.com', '{\"clientId\":\"x\"}', 'u-sso', 'idp-pre13')");
db.run("INSERT INTO organizations (id, user_id, config_id, name, avatar_url, normalized_name) VALUES ('org-pre13', 'u-sso', 'cfg-pre13', 'Example', 'https://example.com/a.png', 'example')");
}
function verify0013Migration(db: any) {
// New columns on sso_providers.
const ssoCols = db
.query("PRAGMA table_info(sso_providers)")
.all() as Array<{ name: string; notnull: number; dflt_value: string | null }>;
const saml = ssoCols.find((c) => c.name === "saml_config");
const domainVerified = ssoCols.find((c) => c.name === "domain_verified");
assert(saml, "Expected sso_providers.saml_config column to exist");
assert(saml.notnull === 0, "Expected saml_config to be nullable");
assert(domainVerified, "Expected sso_providers.domain_verified column to exist");
assert(domainVerified.notnull === 1, "Expected domain_verified to be NOT NULL");
assert(
domainVerified.dflt_value === "true",
`Expected domain_verified DEFAULT true, got ${domainVerified.dflt_value}`,
);
// Pre-existing SSO row picked up the default (1 = true) on domain_verified.
const ssoRow = db
.query("SELECT provider_id, saml_config, domain_verified FROM sso_providers WHERE id = 'sso-pre13'")
.get() as { provider_id: string; saml_config: string | null; domain_verified: number } | null;
assert(ssoRow, "Expected pre-existing OIDC provider row to survive migration");
assert(ssoRow.saml_config === null, `Expected saml_config NULL, got ${ssoRow.saml_config}`);
assert(ssoRow.domain_verified === 1, `Expected domain_verified=1, got ${ssoRow.domain_verified}`);
// Organizations rebuild preserved the seeded row and dropped the inherited
// DEFAULT '' on normalized_name (drizzle reconciles to schema.ts).
const orgRow = db
.query("SELECT id, normalized_name FROM organizations WHERE id = 'org-pre13'")
.get() as { id: string; normalized_name: string } | null;
assert(orgRow, "Expected pre-existing organization row to survive table rebuild");
assert(orgRow.normalized_name === "example", `Expected organization normalized_name preserved, got ${orgRow.normalized_name}`);
const orgCols = db
.query("PRAGMA table_info(organizations)")
.all() as Array<{ name: string; dflt_value: string | null }>;
const normName = orgCols.find((c) => c.name === "normalized_name");
assert(normName, "Expected organizations.normalized_name column to exist");
assert(normName.dflt_value === null, `Expected normalized_name to have no default, got ${normName.dflt_value}`);
}
const MIGRATION_0012_TIMESTAMP = 1774062000000;
const MIGRATION_0013_TIMESTAMP = 1780377747526;
/**
* Reproduce the issue #312 crash state sso_providers already carries
* saml_config / domain_verified before migration 0013 runs (stranded on an
* intermediate build), with __drizzle_migrations recorded only through 0012
* and verify repairDuplicateSsoColumns()/restoreSsoDataAfter0013() let the
* canonical 0013 run while preserving real SAML provider data.
*/
function validateBroken0013Repair() {
const migration0013 = migrations.find((m) => m.entry.tag === "0013_slim_galactus");
if (!migration0013) return; // 0013 not present (shouldn't happen) — nothing to test.
const db = new Database(":memory:");
try {
runMigrations(db, migrations.slice(0, 13)); // 0000-0012
// A real upgraded instance has a __drizzle_migrations table recorded
// through 0012 but not 0013.
db.run(
"CREATE TABLE IF NOT EXISTS `__drizzle_migrations` (id INTEGER PRIMARY KEY AUTOINCREMENT, hash text NOT NULL, created_at numeric)",
);
db.run("INSERT INTO `__drizzle_migrations` (hash, created_at) VALUES ('through-0012', ?)", [
MIGRATION_0012_TIMESTAMP,
]);
// Stranded columns from the intermediate build.
db.run("ALTER TABLE sso_providers ADD saml_config text");
db.run("ALTER TABLE sso_providers ADD domain_verified integer DEFAULT true NOT NULL");
db.run("INSERT INTO users (id, email, username, name) VALUES ('u1', 'u1@example.com', 'u1', 'User One')");
const samlJson = '{"entryPoint":"https://idp.example.com/sso","cert":"ABC123"}';
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('oidc1', 'https://idp', 'a.com', '{}', 'u1', 'p-oidc', NULL, 1)",
);
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('saml1', 'https://idp', 'b.com', '{}', 'u1', 'p-saml', ?, 1)",
[samlJson],
);
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('unv1', 'https://idp', 'c.com', '{}', 'u1', 'p-unv', NULL, 0)",
);
const preserved = repairDuplicateSsoColumns(db);
const colsAfterRepair = (db.query("PRAGMA table_info(sso_providers)").all() as TableInfoRow[]).map(
(c) => c.name,
);
assert(!colsAfterRepair.includes("saml_config"), "Expected repair to drop stranded saml_config column");
assert(
!colsAfterRepair.includes("domain_verified"),
"Expected repair to drop stranded domain_verified column",
);
const preservedIds = preserved.map((r) => r.id).sort();
assert(
preservedIds.length === 2 && preservedIds[0] === "saml1" && preservedIds[1] === "unv1",
`Expected SAML + unverified rows to be preserved, got ${JSON.stringify(preservedIds)}`,
);
// The canonical 0013 must now run without a duplicate-column error.
runMigration(db, migration0013);
restoreSsoDataAfter0013(db, preserved);
const rows = db
.query("SELECT id, saml_config, domain_verified FROM sso_providers ORDER BY id")
.all() as Array<{ id: string; saml_config: string | null; domain_verified: number }>;
const byId = Object.fromEntries(rows.map((r) => [r.id, r]));
assert(byId.oidc1.saml_config === null, "Expected OIDC provider saml_config to remain NULL");
assert(byId.oidc1.domain_verified === 1, "Expected OIDC provider domain_verified default 1");
assert(byId.saml1.saml_config === samlJson, "Expected SAML provider config to be preserved");
assert(byId.saml1.domain_verified === 1, "Expected SAML provider domain_verified preserved as 1");
assert(byId.unv1.saml_config === null, "Expected unverified provider saml_config NULL");
assert(byId.unv1.domain_verified === 0, "Expected explicit domain_verified=0 to be preserved");
// Idempotency: 0013 is now applied, so a re-run of the repair is a no-op.
db.run("INSERT INTO `__drizzle_migrations` (hash, created_at) VALUES ('through-0013', ?)", [
MIGRATION_0013_TIMESTAMP,
]);
const secondPass = repairDuplicateSsoColumns(db);
assert(secondPass.length === 0, "Expected repair to no-op once migration 0013 is recorded");
const colsAfterSecondPass = (
db.query("PRAGMA table_info(sso_providers)").all() as TableInfoRow[]
).map((c) => c.name);
assert(
colsAfterSecondPass.includes("saml_config") && colsAfterSecondPass.includes("domain_verified"),
"Expected columns to remain intact on the no-op second pass",
);
} finally {
db.close();
}
}
const latestUpgradeFixtures: Record<string, UpgradeFixture> = {
"0009_nervous_tyger_tiger": {
seed: seedPre0009Database,
@@ -201,6 +401,14 @@ const latestUpgradeFixtures: Record<string, UpgradeFixture> = {
seed: seedPre0011Database,
verify: verify0011Migration,
},
"0012_oauth_provider_migration": {
seed: seedPre0012Database,
verify: verify0012Migration,
},
"0013_slim_galactus": {
seed: seedPre0013Database,
verify: verify0013Migration,
},
};
function lintMigrations(selectedMigrations: Migration[]) {
@@ -251,8 +459,11 @@ function validateMigrations() {
upgradeDb.close();
}
// Exercise the runtime repair for the issue #312 duplicate-column crash.
validateBroken0013Repair();
console.log(
`Validated ${migrations.length} migrations from scratch and upgrade path for ${latestMigration.entry.tag}.`,
`Validated ${migrations.length} migrations from scratch and upgrade path for ${latestMigration.entry.tag}, plus the #312 SSO-column repair.`,
);
}
+97 -7
View File
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useState, useRef } from 'react';
import { Button } from '@/components/ui/button';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter } from 'lucide-react';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter, StopCircle } from 'lucide-react';
import {
DropdownMenu,
DropdownMenuContent,
@@ -85,6 +85,8 @@ export function ActivityLog() {
const [activities, setActivities] = useState<MirrorJobWithKey[]>([]);
const [isInitialLoading, setIsInitialLoading] = useState(false);
const [showCleanupDialog, setShowCleanupDialog] = useState(false);
const [showCancelPendingDialog, setShowCancelPendingDialog] = useState(false);
const [isCancelPendingLoading, setIsCancelPendingLoading] = useState(false);
// Ref to track if component is mounted to prevent state updates after unmount
const isMountedRef = useRef(true);
@@ -354,6 +356,40 @@ export function ActivityLog() {
setShowCleanupDialog(false);
};
const confirmCancelPending = async () => {
if (!user?.id) return;
try {
setIsCancelPendingLoading(true);
setShowCancelPendingDialog(false);
const response = await fetch(withBase('/api/job/cancel-pending'), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
});
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: 'Unknown error occurred' }));
throw new Error(errorData.error || `HTTP ${response.status}: ${response.statusText}`);
}
const res = await response.json();
if (res.success) {
toast.success(res.message);
// Refresh to show the new activity log entry
await fetchActivities(false);
} else {
showErrorToast(res.error || 'Failed to cancel pending mirrors.', toast);
}
} catch (error) {
console.error('Error cancelling pending mirrors:', error);
showErrorToast(error, toast);
} finally {
setIsCancelPendingLoading(false);
}
};
// Check if any filters are active
const hasActiveFilters = !!(filter.status || filter.type || filter.name);
const activeFilterCount = [filter.status, filter.type, filter.name].filter(Boolean).length;
@@ -552,11 +588,22 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10 shrink-0"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10 shrink-0"
>
<Trash2 className="h-4 w-4" />
@@ -683,12 +730,24 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
{/* cleanup all activities */}
{/* stop pending mirrors */}
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
{/* clear activity history */}
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10"
>
<Trash2 className="h-4 w-4" />
@@ -709,9 +768,12 @@ export function ActivityLog() {
<Dialog open={showCleanupDialog} onOpenChange={setShowCleanupDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete All Activities</DialogTitle>
<DialogTitle>Clear Activity History</DialogTitle>
<DialogDescription>
Are you sure you want to delete ALL activities? This action cannot be undone and will remove all mirror jobs and events from the database.
This clears the activity <strong>history log</strong> (mirror job records and events) it does not stop
any pending or in-progress mirrors. Repositories keep their current status and the scheduler
will continue to pick up pending work. To stop pending mirrors, use the
&ldquo;Stop Pending Mirrors&rdquo; button instead.
</DialogDescription>
</DialogHeader>
<DialogFooter>
@@ -723,7 +785,35 @@ export function ActivityLog() {
onClick={confirmCleanup}
disabled={isInitialLoading}
>
{isInitialLoading ? 'Deleting...' : 'Delete All Activities'}
{isInitialLoading ? 'Clearing...' : 'Clear History'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* cancel pending mirrors confirmation dialog */}
<Dialog open={showCancelPendingDialog} onOpenChange={setShowCancelPendingDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Stop Pending Mirrors</DialogTitle>
<DialogDescription>
This sets all repositories with status <strong>Imported</strong> or <strong>Failed</strong> to{' '}
<strong>Ignored</strong>, preventing the scheduler from mirroring them automatically.
Repositories that are currently mirroring are not affected.{' '}
You can re-enable individual repositories from the Repositories page.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button variant="outline" onClick={() => setShowCancelPendingDialog(false)}>
Cancel
</Button>
<Button
variant="default"
className="bg-amber-600 hover:bg-amber-700"
onClick={confirmCancelPending}
disabled={isCancelPendingLoading}
>
{isCancelPendingLoading ? 'Stopping...' : 'Stop Pending Mirrors'}
</Button>
</DialogFooter>
</DialogContent>
@@ -927,6 +927,26 @@ export function GitHubMirrorSettings({
</p>
</div>
</div>
<div className="flex items-start space-x-3">
<Checkbox
id="skip-personal-repos"
checked={advancedOptions.skipPersonalRepos ?? false}
onCheckedChange={(checked) => handleAdvancedChange('skipPersonalRepos', !!checked)}
/>
<div className="space-y-0.5 flex-1">
<Label
htmlFor="skip-personal-repos"
className="text-sm font-normal cursor-pointer flex items-center gap-2"
>
<Users className="h-3.5 w-3.5" />
Skip personal repositories (only mirror organization repos)
</Label>
<p className="text-xs text-muted-foreground">
Exclude repositories owned by your personal GitHub account; only mirror repos belonging to organizations
</p>
</div>
</div>
</div>
</div>
</div>
+15 -27
View File
@@ -90,41 +90,29 @@ export default function ConsentPage() {
const handleConsent = async (accept: boolean) => {
setIsSubmitting(true);
try {
// The OAuth provider redirected here with the authorization request in
// the query string (client_id, scope, code). Hand that back via
// `oauth_query` so the provider can resume the flow. It validates the
// redirect URI server-side and returns the URL to navigate to — either
// with an authorization code (accept) or an error (deny).
const oauthQuery = window.location.search.replace(/^\?/, '');
const result = await authClient.oauth2.consent({
accept,
oauth_query: oauthQuery,
scope: Array.from(selectedScopes).join(' '),
});
if (result.error) {
throw new Error(result.error.message || 'Consent failed');
}
// The consent method should handle the redirect
if (!accept) {
// If denied, redirect back to the application with error
const params = new URLSearchParams(window.location.search);
const redirectUri = params.get('redirect_uri');
if (redirectUri && application) {
// Validate redirect URI against authorized URIs
const authorizedUris = parseRedirectUris(application.redirectURLs);
if (isValidRedirectUri(redirectUri, authorizedUris)) {
try {
// Parse and reconstruct the URL to ensure it's safe
const url = new URL(redirectUri);
url.searchParams.set('error', 'access_denied');
// Safe to redirect - URI has been validated and sanitized
window.location.href = url.toString();
} catch (e) {
console.error('Failed to parse redirect URI:', e);
setError('Invalid redirect URI');
}
} else {
console.error('Unauthorized redirect URI:', redirectUri);
setError('Invalid redirect URI');
}
}
const redirectUri = (result.data as { redirect_uri?: string } | undefined)?.redirect_uri;
if (redirectUri) {
// The redirect URI is produced and validated server-side.
window.location.href = redirectUri;
} else if (!accept) {
// No redirect target returned on denial — return to the app.
window.location.href = '/';
}
} catch (error) {
showErrorToast(error, toast);
+189 -106
View File
@@ -98,6 +98,8 @@ export default function Repository() {
const [repoToDelete, setRepoToDelete] = useState<Repository | null>(null);
const [isDeleteRepoDialogOpen, setIsDeleteRepoDialogOpen] = useState(false);
const [isDeletingRepo, setIsDeletingRepo] = useState(false);
const [isBulkDeleteDialogOpen, setIsBulkDeleteDialogOpen] = useState(false);
const [isDeletingBulk, setIsDeletingBulk] = useState(false);
// Create a stable callback using useCallback
const handleNewMessage = useCallback((data: MirrorJob) => {
@@ -919,11 +921,9 @@ export default function Repository() {
setIsDeletingRepo(true);
try {
const response = await apiRequest<{ success: boolean; error?: string }>(
`/repositories/${repoToDelete.id}`,
{
method: "DELETE",
}
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [repoToDelete.id] }) }
);
if (response.success) {
@@ -941,6 +941,30 @@ export default function Repository() {
}
};
const handleBulkDelete = async () => {
if (!user || !user.id) return;
setIsDeletingBulk(true);
try {
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [...selectedRepoIds] }) }
);
if (response.success) {
const count = response.deleted ?? selectedRepoIds.size;
toast.success(`Removed ${count} ${count === 1 ? "repository" : "repositories"} from Gitea Mirror.`);
setSelectedRepoIds(new Set());
await fetchRepositories(false);
} else {
showErrorToast(response.error || "Failed to delete repositories", toast);
}
} catch (error) {
showErrorToast(error, toast);
} finally {
setIsDeletingBulk(false);
setIsBulkDeleteDialogOpen(false);
}
};
// Determine what actions are available for selected repositories
const getAvailableActions = () => {
if (selectedRepoIds.size === 0) return [];
@@ -977,7 +1001,9 @@ export default function Repository() {
if (selectedRepos.some(repo => repo.status === "ignored")) {
actions.push('include');
}
actions.push('delete');
return actions;
};
@@ -994,6 +1020,7 @@ export default function Repository() {
retry: selectedRepos.filter(repo => repo.status === "failed").length,
ignore: selectedRepos.filter(repo => repo.status !== "ignored").length,
include: selectedRepos.filter(repo => repo.status === "ignored").length,
delete: selectedRepos.length,
};
};
@@ -1312,111 +1339,122 @@ export default function Repository() {
</Button>
</div>
{/* Bulk actions on desktop - integrated into the same line */}
{/* Mirror All action */}
<div className="flex items-center gap-2 border-l pl-4">
{selectedRepoIds.size === 0 ? (
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
) : (
<>
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
</>
)}
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
</div>
</div>
</div>
{/* Desktop: Bulk actions row - shown when repos are selected */}
{selectedRepoIds.size > 0 && (
<div className="hidden sm:flex items-center gap-2 flex-wrap">
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
<Button
variant="destructive"
size="default"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
)}
{/* Action buttons for mobile - only show when items are selected */}
{selectedRepoIds.size > 0 && (
<div className="flex items-center gap-2 flex-wrap sm:hidden">
@@ -1506,6 +1544,16 @@ export default function Repository() {
Include
</Button>
)}
<Button
variant="destructive"
size="sm"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
</div>
)}
@@ -1586,6 +1634,41 @@ export default function Repository() {
</DialogContent>
</Dialog>
<Dialog
open={isBulkDeleteDialogOpen}
onOpenChange={(open) => {
if (!open && !isDeletingBulk) setIsBulkDeleteDialogOpen(false);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Remove {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"} from Gitea Mirror?</DialogTitle>
<DialogDescription>
These repositories will be deleted from Gitea Mirror only. Any mirrors on Gitea will remain untouched; remove them manually in Gitea if needed.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => setIsBulkDeleteDialogOpen(false)}
disabled={isDeletingBulk}
>
Cancel
</Button>
<Button variant="destructive" onClick={handleBulkDelete} disabled={isDeletingBulk}>
{isDeletingBulk ? (
<LoaderCircle className="h-4 w-4 animate-spin" />
) : (
<span className="flex items-center gap-2">
<Trash2 className="h-4 w-4" />
Delete {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"}
</span>
)}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog
open={isDeleteRepoDialogOpen}
onOpenChange={(open) => {
+57 -12
View File
@@ -99,6 +99,7 @@ export default function RepositoryTable({
onDismissSync,
}: RepositoryTableProps) {
const tableParentRef = useRef<HTMLDivElement>(null);
const lastSelectedIndexRef = useRef<number | null>(null);
const { giteaConfig } = useGiteaConfig();
const handleUpdateDestination = async (repoId: string, newDestination: string | null) => {
@@ -235,6 +236,7 @@ export default function RepositoryTable({
// Selection handlers
const handleSelectAll = (checked: boolean) => {
lastSelectedIndexRef.current = null;
if (checked) {
const allIds = new Set(
visibleRepositories
@@ -247,7 +249,7 @@ export default function RepositoryTable({
}
};
const handleSelectRepo = (repoId: string, checked: boolean) => {
const handleSelectRepo = (repoId: string, checked: boolean, index?: number) => {
const newSelection = new Set(selectedRepoIds);
if (checked) {
newSelection.add(repoId);
@@ -255,6 +257,32 @@ export default function RepositoryTable({
newSelection.delete(repoId);
}
onSelectionChange(newSelection);
if (index !== undefined) {
lastSelectedIndexRef.current = index;
}
};
const handleShiftRangeSelect = (currentIndex: number) => {
const lastIndex = lastSelectedIndexRef.current;
if (lastIndex === null) {
const repo = visibleRepositories[currentIndex];
if (repo?.id) {
const newSelection = new Set(selectedRepoIds);
newSelection.add(repo.id);
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
}
return;
}
const start = Math.min(lastIndex, currentIndex);
const end = Math.max(lastIndex, currentIndex);
const newSelection = new Set(selectedRepoIds);
for (let i = start; i <= end; i++) {
const id = visibleRepositories[i]?.id;
if (id) newSelection.add(id);
}
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
};
const isAllSelected =
@@ -263,7 +291,7 @@ export default function RepositoryTable({
const isPartiallySelected = selectedRepoIds.size > 0 && !isAllSelected;
// Mobile card layout for repository
const RepositoryCard = ({ repo }: { repo: Repository }) => {
const RepositoryCard = ({ repo, index }: { repo: Repository; index: number }) => {
const isLoading = repo.id ? loadingRepoIds.has(repo.id) : false;
const isSelected = repo.id ? selectedRepoIds.has(repo.id) : false;
const giteaUrl = getGiteaRepoUrl(repo);
@@ -274,12 +302,21 @@ export default function RepositoryTable({
<div className="flex flex-col gap-3">
{/* Header with checkbox and repo name */}
<div className="flex items-start gap-3">
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
<div
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(index);
}
}}
>
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean, index)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
</div>
<div className="flex-1 min-w-0">
<h3 className="font-medium text-base truncate">{repo.name}</h3>
<div className="flex items-center gap-2 mt-1 flex-wrap">
@@ -635,8 +672,8 @@ export default function RepositoryTable({
</div>
{/* Repository cards */}
{visibleRepositories.map((repo) => (
<RepositoryCard key={repo.id} repo={repo} />
{visibleRepositories.map((repo, index) => (
<RepositoryCard key={repo.id} repo={repo} index={index} />
))}
</div>
@@ -701,10 +738,18 @@ export default function RepositoryTable({
className="h-[65px] flex items-center justify-between bg-transparent border-b hover:bg-muted/50"
>
{/* Checkbox */}
<div className="h-full p-3 flex items-center justify-center flex-[0.3]">
<div
className="h-full p-3 flex items-center justify-center flex-[0.3]"
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(virtualRow.index);
}
}}
>
<Checkbox
checked={repo.id ? selectedRepoIds.has(repo.id) : false}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked)}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked, virtualRow.index)}
aria-label={`Select ${repo.name}`}
/>
</div>
+2 -2
View File
@@ -1,6 +1,6 @@
import "@/lib/polyfills/buffer";
import { createAuthClient } from "better-auth/react";
import { oidcClient } from "better-auth/client/plugins";
import { oauthProviderClient } from "@better-auth/oauth-provider/client";
import { ssoClient } from "@better-auth/sso/client";
import type { Session as BetterAuthSession, User as BetterAuthUser } from "better-auth";
import { withBase } from "@/lib/base-path";
@@ -41,7 +41,7 @@ export const authClient = createAuthClient({
})(),
basePath: withBase('/api/auth'), // Explicitly set the base path
plugins: [
oidcClient(),
oauthProviderClient(),
ssoClient(),
],
});
+97 -11
View File
@@ -1,6 +1,7 @@
import { betterAuth } from "better-auth";
import { drizzleAdapter } from "better-auth/adapters/drizzle";
import { oidcProvider } from "better-auth/plugins";
import { jwt } from "better-auth/plugins";
import { oauthProvider } from "@better-auth/oauth-provider";
import { sso } from "@better-auth/sso";
import { db, users } from "./db";
import * as schema from "./db/schema";
@@ -74,6 +75,23 @@ export async function resolveTrustedOrigins(request?: Request): Promise<string[]
return uniqueOrigins;
}
/**
* Resolves the Better Auth logger level from BETTER_AUTH_LOG_LEVEL.
* Returns undefined for unset/invalid values so Better Auth falls back
* to its built-in default ("warn"). "success" is intentionally excluded
* it is an output level, not a valid threshold.
*/
function resolveAuthLogLevel(): "debug" | "info" | "warn" | "error" | undefined {
const raw = process.env.BETTER_AUTH_LOG_LEVEL?.trim().toLowerCase();
if (raw === "debug" || raw === "info" || raw === "warn" || raw === "error") {
return raw;
}
if (raw) {
console.warn(`Invalid BETTER_AUTH_LOG_LEVEL: "${raw}", using default ("warn")`);
}
return undefined;
}
export const auth = betterAuth({
// Database configuration
database: drizzleAdapter(db, {
@@ -85,6 +103,17 @@ export const auth = betterAuth({
// Secret for signing tokens
secret: process.env.BETTER_AUTH_SECRET,
// Logger configuration.
//
// Better Auth ships its own logger (it does NOT read the `DEBUG` env
// var / the `debug` npm package — `DEBUG=better-auth:*` is a no-op).
// The default level is "warn", so SSO/OIDC debug and info messages
// are hidden out of the box. Set BETTER_AUTH_LOG_LEVEL=debug to surface
// the full sign-in / callback trace when troubleshooting SSO.
logger: {
level: resolveAuthLogLevel(),
},
// Base URL configuration - use the primary URL (Better Auth only supports single baseURL)
baseURL: (() => {
const url = process.env.BETTER_AUTH_URL;
@@ -154,26 +183,71 @@ export const auth = betterAuth({
},
},
// Account linking configuration.
//
// Lets a user who first registered with email/password sign in through SSO
// and land on the *same* account, instead of being bounced back to /login.
// Better Auth's auto-link path (link-account.mjs) refuses unless BOTH sides
// pass:
// - upstream: provider is "trusted" (either listed in `trustedProviders`
// or the SSO plugin marks it trusted via domainVerified +
// domain match) OR userInfo.emailVerified === true
// - local: existing user is verified (or requireLocalEmailVerified=false)
//
// We don't wire an email-verification flow, so the local admin always has
// emailVerified=false — `requireLocalEmailVerified: false` is required.
//
// We deliberately do NOT use the catch-all `trustedProviders` list (which
// would blanket-trust every registered IdP). Instead the SSO provider's
// own `domainVerified` flag — set to true at registration time, scoped to
// the operator-supplied `domain` — gates linking. The SSO plugin enforces
// `validateEmailDomain(userInfo.email, provider.domain)` on top of it, so
// a sign-in is only auto-linked when (a) the operator vouched for the IdP
// by registering it, and (b) the user's email actually belongs to the
// domain that was vouched for. Cross-domain claims from a compromised or
// permissive IdP do not silently absorb local accounts. (Same-domain
// claims still require the operator to trust their IdP's identity model.)
account: {
accountLinking: {
enabled: true,
requireLocalEmailVerified: false,
// Keep the default (false): never link accounts whose emails differ.
allowDifferentEmails: false,
},
},
// Plugins configuration
plugins: [
// OIDC Provider plugin - allows this app to act as an OIDC provider
oidcProvider({
// JWT plugin — provides the JWKS keypair that the OAuth provider uses to
// sign OIDC id_tokens with an asymmetric key (what most relying parties
// expect). Required by @better-auth/oauth-provider unless disableJwtPlugin
// is set. Keys are persisted in the `jwks` table.
jwt(),
// OAuth 2.1 / OIDC Provider — allows this app to act as an identity
// provider for other applications. Replaces the deprecated `oidcProvider`
// plugin (which Better Auth will remove in a future release). Client and
// consent records now live in the oauth_clients / oauth_consents tables.
oauthProvider({
loginPage: withBase("/login"),
consentPage: withBase("/oauth/consent"),
// Allow dynamic client registration for flexibility
allowDynamicClientRegistration: true,
// Note: trustedClients would be configured here if Better Auth supports it
// For now, we'll use dynamic registration
// Customize user info claims based on scopes
getAdditionalUserInfoClaim: (user, scopes) => {
// Mirror the old getAdditionalUserInfoClaim: expose our extra `username`
// field on both the userinfo endpoint and the id_token when the
// "profile" scope is granted.
customUserInfoClaims: ({ user, scopes }) => {
const claims: Record<string, any> = {};
if (scopes.includes("profile")) {
claims.username = user.username;
}
if (scopes.includes("profile")) claims.username = (user as any).username;
return claims;
},
customIdTokenClaims: ({ user, scopes }) => {
const claims: Record<string, any> = {};
if (scopes.includes("profile")) claims.username = (user as any).username;
return claims;
},
}),
// SSO plugin - allows users to authenticate with external OIDC providers
sso({
// Provision new users when they sign in with SSO
@@ -205,6 +279,18 @@ export const auth = betterAuth({
disableImplicitSignUp: false,
// Trust email_verified claims from the upstream provider so we can link by matching email
trustEmailVerified: true,
// Surface `domainVerified` to the model so account-linking can read it.
//
// Better Auth's adapter transformOutput (factory.mjs) only copies fields
// that are declared in the plugin's model schema; the SSO plugin only
// declares `domainVerified` when this option is enabled. Without it the
// column is in the DB but stripped from the returned provider object, so
// the trust check `"domainVerified" in provider` is silently false and
// sign-ins land on /?error=UNKNOWN. We don't use the DNS-based
// verify-domain flow this option also exposes — we set
// `domainVerified: true` directly in src/pages/api/auth/sso/register.ts
// after the plugin's create, scoped by the operator-supplied `domain`.
domainVerification: { enabled: true },
}),
// Header / forward authentication bridge. Exposes
+13 -2
View File
@@ -3,6 +3,7 @@ import { drizzle } from "drizzle-orm/bun-sqlite";
import fs from "fs";
import path from "path";
import { migrate } from "drizzle-orm/bun-sqlite/migrator";
import { repairDuplicateSsoColumns, restoreSsoDataAfter0013 } from "./migration-repairs";
// Skip database initialization in test environment
let db: ReturnType<typeof drizzle>;
@@ -95,9 +96,17 @@ if (process.env.NODE_ENV !== "test") {
// Fix any migrations that were recorded but actually failed (e.g. v3.13.0 bug)
repairFailedMigrations();
// Fix the v3.17.0 duplicate-column crash: reconcile stranded sso_providers
// columns so migration 0013 can run (see #312). Returns data to re-apply
// once 0013 has re-added the columns.
const preservedSsoData = repairDuplicateSsoColumns(sqlite);
// Run migrations using Drizzle migrate function
migrate(db, { migrationsFolder: "./drizzle" });
// Re-apply any SSO provider data preserved by the repair above.
restoreSsoDataAfter0013(sqlite, preservedSsoData);
console.log("✅ Database migrations completed successfully");
} catch (error) {
console.error("❌ Error running migrations:", error);
@@ -123,9 +132,11 @@ export {
accounts,
verificationTokens,
verifications,
oauthApplications,
oauthClients,
oauthAccessTokens,
oauthConsent,
oauthRefreshTokens,
oauthConsents,
jwkss,
ssoProviders,
rateLimits
} from "./schema";
+117
View File
@@ -0,0 +1,117 @@
import type { Database } from "bun:sqlite";
/**
* Pre-migration repairs that reconcile a database into the exact shape Drizzle's
* migrator expects, so a previously-failed migration can complete on the next
* boot. These run BEFORE `migrate()` and are deliberately defensive: any failure
* is logged and swallowed so they never make a recoverable database worse.
*/
/** Migration 0013 journal timestamp (from drizzle/meta/_journal.json, idx 13). */
const MIGRATION_0013_TIMESTAMP = 1780377747526;
export type PreservedSsoRow = {
id: string;
saml_config?: string | null;
domain_verified?: number;
};
/**
* Repair the v3.17.0 (PR #307) "duplicate column name: saml_config" crash loop
* reported in issue #312.
*
* Some instances ended up with `sso_providers.saml_config` / `domain_verified`
* already present BEFORE migration 0013 ran the columns were declared in
* schema.ts and entered the DB via `db:push` or an SSO-register round-trip on an
* intermediate build, while `__drizzle_migrations` never recorded a 0013 row.
*
* Migration 0013 runs as a single transaction (organizations rebuild + the two
* `ALTER TABLE sso_providers ADD ...`). The ADD hits the pre-existing column,
* throws "duplicate column", and rolls back the ENTIRE transaction so 0013 is
* never recorded and is retried, failing identically, on every boot.
*
* This is the mirror image of the 0009 repair in index.ts (record present,
* column missing): here the column is present but the record is missing. We
* reconcile `sso_providers` back to its true pre-0013 shape so the canonical
* 0013 can run in full (the organizations rebuild MUST NOT be skipped),
* preserving any real SAML provider config across the drop/re-add.
*
* Returns the rows whose values must be re-applied by {@link restoreSsoDataAfter0013}
* once 0013 has re-added the columns. Returns an empty array when there is
* nothing to do (fresh install, clean upgrade, or genuine pre-0013 shape).
*/
export function repairDuplicateSsoColumns(sqlite: Database): PreservedSsoRow[] {
try {
const migrationsTableExists = sqlite
.query("SELECT name FROM sqlite_master WHERE type='table' AND name='__drizzle_migrations'")
.get();
// Fresh install — no migrations recorded yet, vanilla migrate() handles it.
if (!migrationsTableExists) return [];
// 0013 already recorded (clean upgrade / already healed) — nothing to do.
const alreadyApplied = sqlite
.query("SELECT 1 FROM __drizzle_migrations WHERE created_at >= ? LIMIT 1")
.get(MIGRATION_0013_TIMESTAMP);
if (alreadyApplied) return [];
const ssoExists = sqlite
.query("SELECT name FROM sqlite_master WHERE type='table' AND name='sso_providers'")
.get();
if (!ssoExists) return [];
const cols = sqlite.query("PRAGMA table_info(sso_providers)").all() as { name: string }[];
const hasSaml = cols.some((c) => c.name === "saml_config");
const hasDomainVerified = cols.some((c) => c.name === "domain_verified");
// Genuine pre-0013 shape — let migration 0013 add the columns as-is.
if (!hasSaml && !hasDomainVerified) return [];
console.log(
"🔧 Detected stranded SSO columns (migration 0013 not recorded). Reconciling sso_providers so 0013 can run...",
);
// Preserve any real data before dropping. SAML providers store JSON config
// in saml_config; domain_verified may have been explicitly set to false.
const selectCols = ["id"];
if (hasSaml) selectCols.push("saml_config");
if (hasDomainVerified) selectCols.push("domain_verified");
const preserved = sqlite
.query(`SELECT ${selectCols.join(", ")} FROM sso_providers`)
.all() as PreservedSsoRow[];
// SQLite >= 3.35 (bun:sqlite ships much newer) supports DROP COLUMN.
if (hasSaml) sqlite.run("ALTER TABLE sso_providers DROP COLUMN saml_config");
if (hasDomainVerified) sqlite.run("ALTER TABLE sso_providers DROP COLUMN domain_verified");
// Only rows whose values differ from the 0013 defaults (saml_config NULL,
// domain_verified true/1) need restoring after the columns are re-added.
return preserved.filter(
(r) => (hasSaml && r.saml_config != null) || (hasDomainVerified && r.domain_verified === 0),
);
} catch (error) {
console.warn("⚠️ SSO column repair check failed (non-fatal):", error);
return [];
}
}
/**
* Re-apply the SSO provider values preserved by {@link repairDuplicateSsoColumns}
* once migration 0013 has re-added saml_config / domain_verified with their
* defaults (saml_config NULL, domain_verified = 1). No-op when nothing was
* preserved (the common OIDC-only case).
*/
export function restoreSsoDataAfter0013(sqlite: Database, preserved: PreservedSsoRow[]): void {
if (preserved.length === 0) return;
try {
const stmt = sqlite.prepare(
"UPDATE sso_providers SET saml_config = ?, domain_verified = ? WHERE id = ?",
);
for (const r of preserved) {
stmt.run(r.saml_config ?? null, r.domain_verified ?? 1, r.id);
}
console.log(`✅ Restored ${preserved.length} preserved SSO provider value(s) after migration 0013.`);
} catch (error) {
console.warn("⚠️ Failed to restore preserved SSO data (non-fatal):", error);
}
}
+110 -46
View File
@@ -34,6 +34,7 @@ export const githubConfigSchema = z.object({
autoMirrorStarred: z.boolean().default(false),
skipStarredIssues: z.boolean().optional(), // Deprecated: kept for backward compatibility, use starredCodeOnly instead
starredDuplicateStrategy: z.enum(["suffix", "prefix", "owner-org"]).default("suffix").optional(),
skipPersonalRepos: z.boolean().default(false),
});
export const backupStrategyEnum = z.enum([
@@ -156,7 +157,9 @@ export const configSchema = z.object({
isActive: z.boolean().default(true),
githubConfig: githubConfigSchema,
giteaConfig: giteaConfigSchema,
// Unused/reserved — stored for future glob support but not currently read
include: z.array(z.string()).default(["*"]),
// Unused/reserved — stored for future glob support but not currently read
exclude: z.array(z.string()).default([]),
scheduleConfig: scheduleConfigSchema,
cleanupConfig: cleanupConfigSchema,
@@ -618,69 +621,121 @@ export const verifications = sqliteTable("verifications", {
// ===== OIDC Provider Tables =====
// OAuth Applications table
export const oauthApplications = sqliteTable("oauth_applications", {
// ===== OAuth 2.1 / OIDC Provider tables (@better-auth/oauth-provider) =====
//
// These back the OAuth/OIDC *provider* feature (gitea-mirror acting as an
// identity provider for other apps). They are managed entirely by Better
// Auth's drizzle adapter, so:
// - the exported binding name must equal the plugin model name pluralized
// under `usePlural: true` (oauthClient -> oauthClients, jwks -> jwkss);
// - the object property names must match the plugin field names (camelCase),
// while the SQL column names may be snake_case;
// - `string[]` and `json` fields are serialized to JSON text by the adapter,
// so they are plain `text` columns here.
//
// Migrated from the deprecated `oidc-provider` plugin (tables
// oauth_applications / oauth_access_tokens / oauth_consent). See the
// accompanying Drizzle migration for the data-preserving upgrade path.
// OAuth clients (replaces the old `oauth_applications` table)
export const oauthClients = sqliteTable("oauth_clients", {
id: text("id").primaryKey(),
clientId: text("client_id").notNull().unique(),
clientSecret: text("client_secret").notNull(),
name: text("name").notNull(),
redirectURLs: text("redirect_urls").notNull(), // Comma-separated list
metadata: text("metadata"), // JSON string
type: text("type").notNull(), // web, mobile, etc
disabled: integer("disabled", { mode: "boolean" }).notNull().default(false),
userId: text("user_id"), // Optional - owner of the application
createdAt: integer("created_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
updatedAt: integer("updated_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
clientSecret: text("client_secret"),
name: text("name"),
disabled: integer("disabled", { mode: "boolean" }).default(false),
skipConsent: integer("skip_consent", { mode: "boolean" }),
enableEndSession: integer("enable_end_session", { mode: "boolean" }),
subjectType: text("subject_type"),
scopes: text("scopes"), // JSON string[]
userId: text("user_id").references(() => users.id),
uri: text("uri"),
icon: text("icon"),
contacts: text("contacts"), // JSON string[]
tos: text("tos"),
policy: text("policy"),
softwareId: text("software_id"),
softwareVersion: text("software_version"),
softwareStatement: text("software_statement"),
redirectUris: text("redirect_uris").notNull(), // JSON string[]
postLogoutRedirectUris: text("post_logout_redirect_uris"), // JSON string[]
tokenEndpointAuthMethod: text("token_endpoint_auth_method"),
grantTypes: text("grant_types"), // JSON string[]
responseTypes: text("response_types"), // JSON string[]
public: integer("public", { mode: "boolean" }),
type: text("type"),
requirePKCE: integer("require_pkce", { mode: "boolean" }),
referenceId: text("reference_id"),
metadata: text("metadata"), // JSON
createdAt: integer("created_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
updatedAt: integer("updated_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
}, (table) => [
index("idx_oauth_applications_client_id").on(table.clientId),
index("idx_oauth_applications_user_id").on(table.userId),
index("idx_oauth_clients_client_id").on(table.clientId),
index("idx_oauth_clients_user_id").on(table.userId),
]);
// OAuth Access Tokens table
// OAuth access tokens
export const oauthAccessTokens = sqliteTable("oauth_access_tokens", {
id: text("id").primaryKey(),
accessToken: text("access_token").notNull(),
refreshToken: text("refresh_token"),
accessTokenExpiresAt: integer("access_token_expires_at", { mode: "timestamp" }).notNull(),
refreshTokenExpiresAt: integer("refresh_token_expires_at", { mode: "timestamp" }),
token: text("token").unique(),
clientId: text("client_id").notNull(),
userId: text("user_id").notNull().references(() => users.id),
scopes: text("scopes").notNull(), // Comma-separated list
createdAt: integer("created_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
updatedAt: integer("updated_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
sessionId: text("session_id"),
userId: text("user_id").references(() => users.id),
referenceId: text("reference_id"),
refreshId: text("refresh_id"),
expiresAt: integer("expires_at", { mode: "timestamp" }),
createdAt: integer("created_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
scopes: text("scopes").notNull(), // JSON string[]
}, (table) => [
index("idx_oauth_access_tokens_access_token").on(table.accessToken),
index("idx_oauth_access_tokens_user_id").on(table.userId),
index("idx_oauth_access_tokens_token").on(table.token),
index("idx_oauth_access_tokens_client_id").on(table.clientId),
index("idx_oauth_access_tokens_user_id").on(table.userId),
]);
// OAuth Consent table
export const oauthConsent = sqliteTable("oauth_consent", {
// OAuth refresh tokens (new in the OAuth 2.1 provider)
export const oauthRefreshTokens = sqliteTable("oauth_refresh_tokens", {
id: text("id").primaryKey(),
userId: text("user_id").notNull().references(() => users.id),
token: text("token").notNull().unique(),
clientId: text("client_id").notNull(),
scopes: text("scopes").notNull(), // Comma-separated list
consentGiven: integer("consent_given", { mode: "boolean" }).notNull(),
createdAt: integer("created_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
updatedAt: integer("updated_at", { mode: "timestamp" })
.notNull()
.default(sql`(unixepoch())`),
sessionId: text("session_id"),
userId: text("user_id").notNull().references(() => users.id),
referenceId: text("reference_id"),
expiresAt: integer("expires_at", { mode: "timestamp" }),
createdAt: integer("created_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
revoked: integer("revoked", { mode: "timestamp" }),
authTime: integer("auth_time", { mode: "timestamp" }),
scopes: text("scopes").notNull(), // JSON string[]
}, (table) => [
index("idx_oauth_consent_user_id").on(table.userId),
index("idx_oauth_consent_client_id").on(table.clientId),
index("idx_oauth_consent_user_client").on(table.userId, table.clientId),
index("idx_oauth_refresh_tokens_token").on(table.token),
index("idx_oauth_refresh_tokens_client_id").on(table.clientId),
index("idx_oauth_refresh_tokens_user_id").on(table.userId),
]);
// OAuth consent records
export const oauthConsents = sqliteTable("oauth_consents", {
id: text("id").primaryKey(),
clientId: text("client_id").notNull(),
userId: text("user_id").references(() => users.id),
referenceId: text("reference_id"),
scopes: text("scopes").notNull(), // JSON string[]
createdAt: integer("created_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
updatedAt: integer("updated_at", { mode: "timestamp" }).default(sql`(unixepoch())`),
}, (table) => [
index("idx_oauth_consents_client_id").on(table.clientId),
index("idx_oauth_consents_user_id").on(table.userId),
]);
// JWKS keypairs for signing OIDC id_tokens (better-auth `jwt` plugin).
// Model name "jwks" pluralizes to the binding name "jwkss" under usePlural,
// while the physical table stays "jwks".
export const jwkss = sqliteTable("jwks", {
id: text("id").primaryKey(),
publicKey: text("public_key").notNull(),
privateKey: text("private_key").notNull(),
createdAt: integer("created_at", { mode: "timestamp" }).notNull().default(sql`(unixepoch())`),
expiresAt: integer("expires_at", { mode: "timestamp" }),
});
// ===== SSO Provider Tables =====
// SSO Providers table
@@ -689,6 +744,15 @@ export const ssoProviders = sqliteTable("sso_providers", {
issuer: text("issuer").notNull(),
domain: text("domain").notNull(),
oidcConfig: text("oidc_config").notNull(), // JSON string with OIDC configuration
// The upgraded @better-auth/sso plugin writes this on every insert (null for OIDC providers).
// Drizzle's adapter rejects unknown fields, so the column must exist.
samlConfig: text("saml_config"),
// Used by the SSO plugin's account-linking trust check: a sign-in is treated
// as trusted when this is true AND the user's email domain matches `domain`
// above. We set this to true on register (see /api/auth/sso/register.ts) so
// domain-scoped auto-linking works out of the box; the column default keeps
// existing rows trusted after upgrade.
domainVerified: integer("domain_verified", { mode: "boolean" }).notNull().default(true),
userId: text("user_id").notNull(), // Admin who created this provider
providerId: text("provider_id").notNull().unique(), // Unique identifier for the provider
organizationId: text("organization_id"), // Optional - if provider is linked to an organization
+2
View File
@@ -285,6 +285,8 @@ export async function initializeConfigFromEnv(): Promise<void> {
starredCodeOnly: envConfig.github.starredCodeOnly ?? existingConfig?.[0]?.githubConfig?.starredCodeOnly ?? false,
autoMirrorStarred: envConfig.github.autoMirrorStarred ?? existingConfig?.[0]?.githubConfig?.autoMirrorStarred ?? false,
starredLists: envConfig.github.starredLists ?? existingConfig?.[0]?.githubConfig?.starredLists ?? [],
// ONLY_MIRROR_ORGS=true maps to skipPersonalRepos: true
skipPersonalRepos: envConfig.github.onlyMirrorOrgs ?? existingConfig?.[0]?.githubConfig?.skipPersonalRepos ?? false,
};
// Build Gitea config
+38 -4
View File
@@ -43,13 +43,18 @@ type SyncDependencies = {
/**
* Enhanced repository information including mirror status
*/
interface GiteaRepoInfo {
export interface GiteaRepoInfo {
id: number;
name: string;
owner: { login: string } | string;
mirror: boolean;
mirror_interval?: string;
clone_url?: string;
// Original migration source URL. Gitea/Forgejo populate this with the
// upstream clone address for migrated/mirrored repos, so it is the
// authoritative way to tell whether an existing mirror points at THIS
// GitHub source (vs. a same-named mirror of a different source).
original_url?: string;
private: boolean;
}
@@ -544,9 +549,12 @@ export async function syncGiteaRepoEnhanced({
// Create backup if strategy says so
if (shouldBackupForStrategy(backupStrategy, forcePushDetected)) {
const cloneUrl =
repoInfo.clone_url ||
`${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
// Always derive the clone URL from the user-configured Gitea URL rather
// than repoInfo.clone_url (which reflects Gitea's ROOT_URL and may be
// unreachable from the app — e.g. Tailscale MagicDNS deployments where
// ROOT_URL resolves externally but the app talks to Gitea on a private
// address).
const cloneUrl = `${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
try {
const backupResult = await createPreSyncBundleBackup({
@@ -886,6 +894,32 @@ export async function syncGiteaRepoEnhanced({
status: "failed",
});
}
} else if (syncError instanceof HttpError && syncError.status === 405) {
// Gitea returns HTTP 405 (with an empty body) when the repository is not
// a pull-mirror in its database — e.g. Gitea auto-disabled the mirror or
// the repo lost its mirror state after a manual edit.
const actionableMessage =
`Gitea reports this repository is not a pull mirror (HTTP 405). ` +
`In Gitea check Settings → Mirror Settings; if the mirror section is ` +
`missing, delete the repository in Gitea and re-mirror it from gitea-mirror.`;
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("failed"),
updatedAt: new Date(),
errorMessage: actionableMessage,
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Sync failed: ${repository.name} is not a pull mirror in Gitea (HTTP 405)`,
details: actionableMessage,
status: "failed",
});
}
throw syncError;
}
+150
View File
@@ -0,0 +1,150 @@
/**
* Unit tests for release reconciliation logic regression for #310.
*
* Root-cause verdict: Theory A (not Theory B).
*
* The old `needsRecreation` check compared published_at-based expected indices
* against Gitea's API order (which mirrors sort by tag-commit-date, not
* published_at). For repos where published_at order permanently disagrees with
* tag-commit-date order, `currentExpectedIdx < nextExpectedIdx` evaluates true
* on every single sync, triggering delete-and-recreate forever.
*
* Discriminating evidence unaconfig_dart fixture:
* v0.1.1 published_at 2024-01-13T23:44 (earlier) expectedOrder index 0
* v0.1.0 published_at 2024-01-16T00:23 (later) expectedOrder index 1
* v0.1.0 tagged 2024-01-13T23:30 (earlier tag commit)
* v0.1.1 tagged 2024-01-13T23:42 (later tag commit)
* Gitea tag-commit order: [v0.1.1, v0.1.0] (v0.1.1 has newer commit)
* Old check: currentExpectedIdx(v0.1.1)=0 < nextExpectedIdx(v0.1.0)=1 TRUE
* needsRecreation fires on every sync, forever.
*
* Theory B (operator inversion) would fire for ALL repos with >1 release, but
* field evidence shows only ~2 of 150 repos are affected ruling it out.
*
* Fix: replaced `needsRecreation` machinery with set-based reconciliation via
* `classifyReleasesForReconciliation`. Releases are created when missing, skipped
* (or PATCH-updated if content drifted) when present. No deletions for ordering.
*/
import { describe, expect, it } from "bun:test";
import { classifyReleasesForReconciliation } from "@/lib/gitea";
describe("classifyReleasesForReconciliation", () => {
describe("normal repo — published_at order matches tag-commit order", () => {
it("creates releases missing in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
expect(toSkip).toEqual([]);
});
it("skips releases already present in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.1.0", "v1.2.0"];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
});
it("creates missing releases while skipping existing ones", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.2.0"]; // v1.1.0 is missing
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.1.0"]);
expect(toSkip).toEqual(["v1.0.0", "v1.2.0"]);
});
it("does NOT produce any deletions — order mismatches are ignored", () => {
// Even if Gitea has them in a different order, the function never suggests deletion
const github = ["v1.0.0", "v1.1.0"];
const gitea = ["v1.1.0", "v1.0.0"]; // reversed order from Gitea
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v1.0.0");
expect(toSkip).toContain("v1.1.0");
});
});
describe("unaconfig_dart regression — published_at order disagrees with tag-commit order (#310)", () => {
// v0.1.0: tagged 2024-01-13T23:30, published_at 2024-01-16T00:23 (published AFTER v0.1.1)
// v0.1.1: tagged 2024-01-13T23:42, published_at 2024-01-13T23:44 (published BEFORE v0.1.0)
//
// Gitea display order (by tag-commit date): [v0.1.1, v0.1.0] (v0.1.1 tagged later)
// GitHub published_at order (oldest first): [v0.1.1, v0.1.0] (v0.1.1 published earlier)
// Wait — in this specific case the orders AGREE. The inversion scenario is:
// GitHub sorts descending (newest first): [v0.1.0, v0.1.1]
// Gitea API returns by tag-commit DESC: [v0.1.1, v0.1.0]
// Old expectedOrder (ascending published): v0.1.1→0, v0.1.0→1
// Check for [v0.1.1, v0.1.0]: current=v0.1.1(idx=0) < next=v0.1.0(idx=1) → TRUE every time
it("does NOT trigger recreation when published_at order and tag-commit order disagree", () => {
// Both releases already in Gitea (as they would be after first successful sync).
// Old code would fire needsRecreation=true here on every subsequent sync.
// New code: set-based check — both present → toCreate is empty → no deletions.
const github = ["v0.1.0", "v0.1.1"]; // GitHub API returns newest published_at first
const gitea = ["v0.1.1", "v0.1.0"]; // Gitea tag-commit order (v0.1.1 tagged later)
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]); // nothing to create
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v0.1.0");
expect(toSkip).toContain("v0.1.1");
});
it("creates v0.1.0 and v0.1.1 when Gitea has no releases yet (first sync)", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0", "v0.1.1"]);
expect(toSkip).toEqual([]);
});
it("only creates the missing release when one of the two already exists", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea = ["v0.1.1"]; // only v0.1.1 was created so far
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0"]);
expect(toSkip).toEqual(["v0.1.1"]);
});
});
describe("edge cases", () => {
it("handles empty GitHub releases list", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], ["v1.0.0"]);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("handles both lists empty", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], []);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("ignores Gitea releases that are not in the GitHub set (orphans, handled by retention cleanup)", () => {
const github = ["v1.0.0"];
const gitea = ["v1.0.0", "v0.9.0"]; // v0.9.0 is an orphan not in GitHub's limited set
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0"]);
// v0.9.0 not mentioned in either output — handled by retention cleanup, not here
});
});
});
+275 -193
View File
@@ -579,7 +579,27 @@ export const mirrorGithubRepoToGitea = async ({
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
if (
// REUSE-FIRST (issues #315 / #309): before generating any (suffixed) name,
// check whether this exact source is already mirrored — either at the
// recorded mirroredLocation or at the base name. If so, reuse that location
// and route into the "already mirrored" handling below instead of creating
// a duplicate. This must run before generateUniqueRepoName so the names
// converge under concurrency (the in-flight guard then becomes effective).
const { findExistingMirror } = await import("./utils/mirror-source-match");
const existingMirror = await findExistingMirror({
repository,
config,
candidateOwner: repoOwner,
candidateName: repository.name,
});
if (existingMirror) {
repoOwner = existingMirror.owner;
targetRepoName = existingMirror.repoName;
console.log(
`Reusing existing same-source mirror for ${repository.fullName} at ${repoOwner}/${targetRepoName}`
);
} else if (
repository.isStarred &&
config.githubConfig &&
(config.githubConfig.starredReposMode || "dedicated-org") === "dedicated-org"
@@ -594,6 +614,7 @@ export const mirrorGithubRepoToGitea = async ({
githubOwner,
fullName: repository.fullName,
strategy: config.githubConfig.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
if (targetRepoName !== repository.name) {
@@ -643,45 +664,72 @@ export const mirrorGithubRepoToGitea = async ({
strategy: "delete", // Can be configured: "skip", "delete", or "rename"
});
} else if (existingRepoInfo?.mirror) {
console.log(
`Repository ${targetRepoName} already exists in Gitea under ${repoOwner}. Updating database status.`
);
// PHANTOM-FORK GUARD (#309): a mirror at this name is only "ours" if it
// mirrors THIS source. existingMirror short-circuits the check
// because findExistingMirror already confirmed the source match.
const { isMirrorOfSource } = await import("./utils/mirror-source-match");
const sameSource =
!!existingMirror ||
isMirrorOfSource(existingRepoInfo, repository.cloneUrl);
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
if (!sameSource) {
// A different source occupies this name. Treat as a genuine collision:
// generate a unique name and fall through to create a separate mirror.
console.warn(
`[Mirror] ${repoOwner}/${targetRepoName} is a mirror of a different source. ` +
`Generating a unique name for ${repository.fullName} to avoid overwriting it.`
);
targetRepoName = await generateUniqueRepoName({
config,
orgName: repoOwner,
baseName: repository.name,
githubOwner: repository.fullName.split("/")[0],
fullName: repository.fullName,
strategy: config.githubConfig?.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
// expectedLocation is recomputed below before the "mirroring" write.
} else {
console.log(
`Repository ${targetRepoName} already exists in Gitea under ${repoOwner}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
// Append log for "mirrored" status
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea`,
details: `Repository ${repository.name} was found to already exist in Gitea under ${repoOwner} and database status was updated.`,
status: "mirrored",
});
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
console.log(
`Repository ${repository.name} database status updated to mirrored`
);
return;
// Append log for "mirrored" status
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea`,
details: `Repository ${repository.name} was found to already exist in Gitea under ${repoOwner} and database status was updated.`,
status: "mirrored",
});
console.log(
`Repository ${repository.name} database status updated to mirrored`
);
return;
}
} else {
console.warn(
`[Mirror] Repository ${repoOwner}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
@@ -689,6 +737,10 @@ export const mirrorGithubRepoToGitea = async ({
}
}
// Recompute the target location in case a phantom-fork collision above
// forced a renamed target after the initial expectedLocation was derived.
const targetLocation = `${repoOwner}/${targetRepoName}`;
console.log(`Mirroring repository ${repository.name}`);
// DOUBLE-CHECK: Final idempotency check right before updating status
@@ -696,7 +748,7 @@ export const mirrorGithubRepoToGitea = async ({
const finalCheck = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
expectedLocation,
expectedLocation: targetLocation,
});
if (finalCheck) {
@@ -714,7 +766,7 @@ export const mirrorGithubRepoToGitea = async ({
.update(repositories)
.set({
status: repoStatusEnum.parse("mirroring"),
mirroredLocation: expectedLocation,
mirroredLocation: targetLocation,
updatedAt: new Date(),
})
.where(eq(repositories.id, repository.id!));
@@ -1177,6 +1229,14 @@ async function isMirroredLocationClaimedInDb({
* Checks both the Gitea instance (HTTP) and the local DB (mirroredLocation)
* to reduce collisions during concurrent batch mirroring.
*
* Source-aware (issues #315 / #309): when a candidate name is already occupied
* by a mirror of THIS SAME GitHub source, the name is REUSED rather than
* suffixed this is what previously caused starred repos to spawn `-owner`,
* `-owner-1`, duplicates on every re-mirror. Suffixing only happens on a
* genuine different-source collision (preserving the #95/#236 cross-owner
* behavior). The per-user DB claim check is retained so two users mirroring the
* same source into a shared org stay separated.
*
* NOTE: This function only checks availability it does NOT claim the name.
* The actual claim happens later when mirroredLocation is written at the
* status="mirroring" DB update, which is protected by a unique partial index
@@ -1189,6 +1249,7 @@ async function generateUniqueRepoName({
githubOwner,
fullName,
strategy,
sourceCloneUrl,
}: {
config: Partial<Config>;
orgName: string;
@@ -1196,6 +1257,10 @@ async function generateUniqueRepoName({
githubOwner: string;
fullName: string;
strategy?: string;
// Source GitHub clone URL, used to decide whether an occupied name belongs to
// THIS repo's mirror (reuse) or a different source (suffix). When omitted,
// behavior degrades to the legacy "any occupant collides" semantics.
sourceCloneUrl?: string;
}): Promise<string> {
if (!fullName?.includes("/")) {
throw new Error(
@@ -1206,33 +1271,55 @@ async function generateUniqueRepoName({
const duplicateStrategy = strategy || "suffix";
const userId = config.userId || "";
// Helper: check both Gitea and local DB for a candidate name
const isNameTaken = async (candidateName: string): Promise<boolean> => {
const { getGiteaRepoInfo } = await import("./gitea-enhanced");
const { classifyCandidateName } = await import("./utils/mirror-source-match");
// Resolve the I/O for a candidate name (Gitea existence, DB claim, repo info)
// and defer the available/reusable/taken decision to the pure, unit-tested
// classifyCandidateName helper.
const classifyName = async (candidateName: string) => {
const existsInGitea = await isRepoPresentInGitea({
config,
owner: orgName,
repoName: candidateName,
});
if (existsInGitea) return true;
// Also check local DB to catch concurrent batch operations
// where another repo claimed this location but hasn't created it in Gitea yet
// A DB claim by a DIFFERENT repo (concurrent batch) always blocks reuse.
let claimedByOther = false;
if (userId) {
const claimedInDb = await isMirroredLocationClaimedInDb({
claimedByOther = await isMirroredLocationClaimedInDb({
userId,
candidateLocation: `${orgName}/${candidateName}`,
excludeFullName: fullName,
});
if (claimedInDb) return true;
}
return false;
// Only fetch repo info when it can actually change the decision (existing,
// same-source candidate that is not DB-claimed by another repo).
const repoInfo =
existsInGitea && sourceCloneUrl && !claimedByOther
? await getGiteaRepoInfo({
config,
owner: orgName,
repoName: candidateName,
})
: null;
return classifyCandidateName({
existsInGitea,
claimedByOther,
repoInfo,
sourceCloneUrl,
});
};
// First check if base name is available
const baseExists = await isNameTaken(baseName);
if (!baseExists) {
// First check the base name — reuse it if it already holds our own mirror.
const baseClass = await classifyName(baseName);
if (baseClass === "available") {
return baseName;
}
if (baseClass === "reusable") {
console.log(`Reusing existing same-source mirror name: ${orgName}/${baseName}`);
return baseName;
}
@@ -1262,9 +1349,14 @@ async function generateUniqueRepoName({
break;
}
const exists = await isNameTaken(candidateName);
const candidateClass = await classifyName(candidateName);
if (!exists) {
if (candidateClass === "reusable") {
console.log(`Reusing existing same-source mirror name: ${orgName}/${candidateName}`);
return candidateName;
}
if (candidateClass === "available") {
console.log(`Found unique name for duplicate starred repo: ${candidateName}`);
return candidateName;
}
@@ -1314,8 +1406,29 @@ export async function mirrorGitHubRepoToGiteaOrg({
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
// The org we will record/reuse for. Stays === orgName on the create path
// (migration uses orgName + giteaOrgId); a reuse hit may repoint it to the
// recorded mirroredLocation's owner for the early-return DB update.
let targetOwner = orgName;
if (
// REUSE-FIRST (issues #315 / #309): reuse an existing same-source mirror
// before generating any suffixed name. See mirrorGithubRepoToGitea for the
// rationale. Routes a hit into the "already mirrored" handling below.
const { findExistingMirror } = await import("./utils/mirror-source-match");
const existingMirror = await findExistingMirror({
repository,
config,
candidateOwner: orgName,
candidateName: repository.name,
});
if (existingMirror) {
targetOwner = existingMirror.owner;
targetRepoName = existingMirror.repoName;
console.log(
`Reusing existing same-source mirror for ${repository.fullName} at ${targetOwner}/${targetRepoName}`
);
} else if (
repository.isStarred &&
config.githubConfig &&
(config.githubConfig.starredReposMode || "dedicated-org") === "dedicated-org"
@@ -1330,6 +1443,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
githubOwner,
fullName: repository.fullName,
strategy: config.githubConfig.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
if (targetRepoName !== repository.name) {
@@ -1340,7 +1454,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
}
// IDEMPOTENCY CHECK: Check if this repo is already being mirrored
const expectedLocation = `${orgName}/${targetRepoName}`;
const expectedLocation = `${targetOwner}/${targetRepoName}`;
const isCurrentlyMirroring = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
@@ -1358,7 +1472,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const isExisting = await isRepoPresentInGitea({
config,
owner: orgName,
owner: targetOwner,
repoName: targetRepoName,
});
@@ -1366,7 +1480,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const { getGiteaRepoInfo, handleExistingNonMirrorRepo } = await import("./gitea-enhanced");
const existingRepoInfo = await getGiteaRepoInfo({
config,
owner: orgName,
owner: targetOwner,
repoName: targetRepoName,
});
@@ -1379,52 +1493,83 @@ export async function mirrorGitHubRepoToGiteaOrg({
strategy: "delete", // Can be configured: "skip", "delete", or "rename"
});
} else if (existingRepoInfo?.mirror) {
console.log(
`Repository ${targetRepoName} already exists in Gitea organization ${orgName}. Updating database status.`
);
// PHANTOM-FORK GUARD (#309): only treat this as "ours" if it mirrors
// THIS source. existingMirror short-circuits because findExistingMirror already
// confirmed the source match.
const { isMirrorOfSource } = await import("./utils/mirror-source-match");
const sameSource =
!!existingMirror ||
isMirrorOfSource(existingRepoInfo, repository.cloneUrl);
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
if (!sameSource) {
// Different source occupies this name: generate a unique name and
// fall through to create a separate mirror under orgName/giteaOrgId.
console.warn(
`[Mirror] ${targetOwner}/${targetRepoName} is a mirror of a different source. ` +
`Generating a unique name for ${repository.fullName} to avoid overwriting it.`
);
targetOwner = orgName;
targetRepoName = await generateUniqueRepoName({
config,
orgName,
baseName: repository.name,
githubOwner: repository.fullName.split("/")[0],
fullName: repository.fullName,
strategy: config.githubConfig?.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
} else {
console.log(
`Repository ${targetRepoName} already exists in Gitea organization ${targetOwner}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${orgName}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: targetOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
// Create a mirror job log entry
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${targetRepoName} already exists in Gitea organization ${orgName}`,
details: `Repository ${targetRepoName} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
status: "mirrored",
});
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${targetOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
console.log(
`Repository ${targetRepoName} database status updated to mirrored in organization ${orgName}`
);
return;
// Create a mirror job log entry
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${targetRepoName} already exists in Gitea organization ${targetOwner}`,
details: `Repository ${targetRepoName} was found to already exist in Gitea organization ${targetOwner} and database status was updated.`,
status: "mirrored",
});
console.log(
`Repository ${targetRepoName} database status updated to mirrored in organization ${targetOwner}`
);
return;
}
} else {
console.warn(
`[Mirror] Repository ${orgName}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
`[Mirror] Repository ${targetOwner}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
);
}
}
// Recompute the target location in case a phantom-fork collision above
// forced a renamed target after the initial expectedLocation was derived.
const targetLocation = `${orgName}/${targetRepoName}`;
console.log(
`Mirroring repository ${repository.fullName} to organization ${orgName} as ${targetRepoName}`
);
@@ -1437,7 +1582,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const finalCheck = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
expectedLocation,
expectedLocation: targetLocation,
});
if (finalCheck) {
@@ -1455,7 +1600,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
.update(repositories)
.set({
status: repoStatusEnum.parse("mirroring"),
mirroredLocation: expectedLocation,
mirroredLocation: targetLocation,
updatedAt: new Date(),
})
.where(eq(repositories.id, repository.id!));
@@ -2515,6 +2660,37 @@ export const mirrorGitRepoIssuesToGitea = async ({
);
};
/**
* Classify a set of GitHub releases against the set already present in Gitea.
*
* Returns:
* - `toCreate`: tag names that exist on GitHub but are missing from Gitea
* - `toSkip`: tag names that already exist in Gitea (will be handled by PATCH-if-content-changed)
*
* Deliberately does NOT return anything to delete based on ordering Gitea mirrors
* order releases by tag-commit date, which can permanently disagree with GitHub's
* published_at order (e.g. unaconfig_dart v0.1.0/v0.1.1 #310). Destroying and
* re-emitting releases for a cosmetic display-order difference is never worth it.
*/
export function classifyReleasesForReconciliation(
githubTagNames: string[],
giteaTagNames: string[]
): { toCreate: string[]; toSkip: string[] } {
const giteaSet = new Set(giteaTagNames);
const toCreate: string[] = [];
const toSkip: string[] = [];
for (const tag of githubTagNames) {
if (giteaSet.has(tag)) {
toSkip.push(tag);
} else {
toCreate.push(tag);
}
}
return { toCreate, toSkip };
}
export async function mirrorGitHubReleasesToGitea({
octokit,
repository,
@@ -2601,23 +2777,10 @@ export async function mirrorGitHubReleasesToGitea({
let mirroredCount = 0;
let skippedCount = 0;
const getReleaseTimestamp = (release: (typeof limitedReleases)[number]) => {
// Use published_at first (when the release was published on GitHub)
// Fall back to created_at (when the git tag was created) only if published_at is missing
// This matches GitHub's sorting behavior and handles cases where multiple tags
// point to the same commit but have different publish dates
const sourceDate = release.published_at ?? release.created_at ?? "";
const timestamp = sourceDate ? new Date(sourceDate).getTime() : 0;
return Number.isFinite(timestamp) ? timestamp : 0;
};
// Process releases in their GitHub API order (newest first by default)
const releasesToProcess = limitedReleases.slice();
// Capture the latest releases, then process them oldest-to-newest so Gitea mirrors keep chronological order
const releasesToProcess = limitedReleases
.slice()
.sort((a, b) => getReleaseTimestamp(b) - getReleaseTimestamp(a))
.sort((a, b) => getReleaseTimestamp(a) - getReleaseTimestamp(b));
console.log(`[Releases] Processing ${releasesToProcess.length} releases in chronological order (oldest to newest by published date)`);
console.log(`[Releases] Processing ${releasesToProcess.length} releases for ${repository.fullName}`);
releasesToProcess.forEach((rel, idx) => {
const publishedDate = new Date(rel.published_at || rel.created_at);
const createdDate = new Date(rel.created_at);
@@ -2627,85 +2790,10 @@ export async function mirrorGitHubReleasesToGitea({
console.log(`[Releases] ${idx + 1}. ${rel.tag_name} - ${dateInfo}`);
});
// Check if existing releases in Gitea are in the wrong order
// If so, we need to delete and recreate them to fix the ordering
let needsRecreation = false;
try {
const existingReleasesResponse = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases?per_page=100`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
).catch(() => null);
if (existingReleasesResponse && existingReleasesResponse.data && Array.isArray(existingReleasesResponse.data)) {
const existingReleases = existingReleasesResponse.data;
if (existingReleases.length > 0) {
console.log(`[Releases] Found ${existingReleases.length} existing releases in Gitea, checking chronological order...`);
// Create a map of tag_name to expected chronological index (0 = oldest, n = newest)
const expectedOrder = new Map<string, number>();
releasesToProcess.forEach((rel, idx) => {
expectedOrder.set(rel.tag_name, idx);
});
// Check if existing releases are in the correct order based on created_unix
// Gitea sorts by created_unix DESC, so newer releases should have higher created_unix values
const releasesThatShouldExist = existingReleases.filter(r => expectedOrder.has(r.tag_name));
if (releasesThatShouldExist.length > 1) {
for (let i = 0; i < releasesThatShouldExist.length - 1; i++) {
const current = releasesThatShouldExist[i];
const next = releasesThatShouldExist[i + 1];
const currentExpectedIdx = expectedOrder.get(current.tag_name)!;
const nextExpectedIdx = expectedOrder.get(next.tag_name)!;
// Since Gitea returns releases sorted by created_unix DESC:
// - Earlier releases in the list should have HIGHER expected indices (newer)
// - Later releases in the list should have LOWER expected indices (older)
if (currentExpectedIdx < nextExpectedIdx) {
console.log(`[Releases] ⚠️ Incorrect ordering detected: ${current.tag_name} (index ${currentExpectedIdx}) appears before ${next.tag_name} (index ${nextExpectedIdx})`);
needsRecreation = true;
break;
}
}
}
if (needsRecreation) {
console.log(`[Releases] ⚠️ Releases are in incorrect chronological order. Will delete and recreate all releases.`);
// Delete all existing releases that we're about to recreate
for (const existingRelease of releasesThatShouldExist) {
try {
console.log(`[Releases] Deleting incorrectly ordered release: ${existingRelease.tag_name}`);
await httpDelete(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
);
} catch (deleteError) {
console.error(`[Releases] Failed to delete release ${existingRelease.tag_name}: ${deleteError instanceof Error ? deleteError.message : String(deleteError)}`);
}
}
console.log(`[Releases] ✅ Deleted ${releasesThatShouldExist.length} releases. Will recreate in correct chronological order.`);
} else {
console.log(`[Releases] ✅ Existing releases are in correct chronological order.`);
}
}
}
} catch (orderCheckError) {
console.warn(`[Releases] Could not verify release order: ${orderCheckError instanceof Error ? orderCheckError.message : String(orderCheckError)}`);
// Continue with normal processing
}
for (const release of releasesToProcess) {
try {
// Check if release already exists (skip check if we just deleted all releases)
const existingReleasesResponse = needsRecreation ? null : await httpGet(
// Always check if release already exists — reconcile by tag set, not by ordering
const existingReleasesResponse = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/tags/${release.tag_name}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
@@ -2842,12 +2930,6 @@ export async function mirrorGitHubReleasesToGitea({
mirroredCount++;
const noteInfo = originalReleaseNote ? ` with ${originalReleaseNote.length} character changelog` : " without changelog";
console.log(`[Releases] Successfully mirrored release: ${release.tag_name}${noteInfo}`);
// Add delay to ensure proper timestamp ordering in Gitea
// Gitea sorts releases by created_unix DESC, and all releases created in quick succession
// will have nearly identical timestamps. The 1-second delay ensures proper chronological order.
console.log(`[Releases] Waiting 1 second to ensure proper timestamp ordering in Gitea...`);
await new Promise(resolve => setTimeout(resolve, 1000));
} catch (error) {
console.error(`[Releases] Failed to mirror release ${release.tag_name}: ${error instanceof Error ? error.message : String(error)}`);
}
+75 -9
View File
@@ -1,15 +1,23 @@
import { describe, expect, test, mock } from "bun:test";
import { getGithubRepositories } from "@/lib/github";
function makeRepo() {
function makeRepo(overrides: Partial<{
name: string;
full_name: string;
ownerLogin: string;
ownerType: string;
fork: boolean;
}> = {}) {
const ownerLogin = overrides.ownerLogin ?? "octo";
const ownerType = overrides.ownerType ?? "User";
return {
name: "demo",
full_name: "octo/demo",
html_url: "https://github.com/octo/demo",
clone_url: "https://github.com/octo/demo.git",
owner: { login: "octo", type: "User" },
name: overrides.name ?? "demo",
full_name: overrides.full_name ?? `${ownerLogin}/${overrides.name ?? "demo"}`,
html_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}`,
clone_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}.git`,
owner: { login: ownerLogin, type: ownerType },
private: false,
fork: false,
fork: overrides.fork ?? false,
has_issues: true,
archived: false,
size: 1,
@@ -23,11 +31,11 @@ function makeRepo() {
};
}
function makeOctokit() {
function makeOctokit(reposToReturn?: ReturnType<typeof makeRepo>[]) {
let captured: Record<string, unknown> | null = null;
const paginate = mock(async (_method: unknown, options?: Record<string, unknown>) => {
captured = options ?? null;
return [makeRepo()];
return reposToReturn ?? [makeRepo()];
});
return {
octokit: {
@@ -98,3 +106,61 @@ describe("getGithubRepositories - affiliation", () => {
}
});
});
describe("getGithubRepositories - skipPersonalRepos", () => {
const personalRepo = makeRepo({ name: "my-lib", ownerLogin: "octo", ownerType: "User" });
const orgRepo = makeRepo({ name: "org-lib", ownerLogin: "my-org", ownerType: "Organization" });
const otherUserRepo = makeRepo({ name: "collab-lib", ownerLogin: "other-user", ownerType: "User" });
test("default false — keeps all repos including personal", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: false } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — drops repos owned by authenticated user", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — keeps repos owned by other users (collaborator repos)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo, otherUserRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
expect(repos.map((r) => r.name)).toContain("collab-lib");
});
test("skipPersonalRepos=true with no owner configured — keeps all repos (safe fallback)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "", skipPersonalRepos: true } as any },
});
// Empty owner means we can't identify the user, so nothing should be dropped
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — unset (undefined) behaves like false", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo" } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
});
+19 -2
View File
@@ -263,10 +263,21 @@ export async function getGithubRepositories({
);
const skipForks = config.githubConfig?.skipForks ?? false;
const skipPersonalRepos = config.githubConfig?.skipPersonalRepos ?? false;
// The authenticated user's login — used to identify personally-owned repos
const authenticatedUserLogin = config.githubConfig?.owner ?? "";
const filteredRepos = repos.filter((repo) => {
const isForkAllowed = !skipForks || !repo.fork;
return isForkAllowed;
// When skipPersonalRepos is true, drop repos owned by the authenticated user
// (owner.type === "User" and owner.login matches the configured GitHub username).
// Org repos have owner.type === "Organization" so they are always kept.
const isPersonalRepo =
skipPersonalRepos &&
authenticatedUserLogin.length > 0 &&
repo.owner.login === authenticatedUserLogin &&
repo.owner.type === "User";
return isForkAllowed && !isPersonalRepo;
});
return filteredRepos.map((repo) => ({
@@ -667,9 +678,11 @@ export async function getGithubStarredListNames({
export async function getGithubOrganizations({
octokit,
config,
skipOrgNames,
}: {
octokit: Octokit;
config: Partial<Config>;
skipOrgNames?: Set<string>;
}): Promise<{ organizations: GitOrg[]; failedOrgs: { name: string; avatarUrl: string; reason: string }[] }> {
try {
const { data: orgs } = await octokit.orgs.listForAuthenticatedUser({
@@ -682,7 +695,7 @@ export async function getGithubOrganizations({
? excludedOrgsEnv.split(",").map((org) => org.trim().toLowerCase())
: [];
// Filter out excluded organizations
// Filter out excluded and user-ignored organizations
const filteredOrgs = orgs.filter((org) => {
if (excludedOrgs.includes(org.login.toLowerCase())) {
console.log(
@@ -690,6 +703,10 @@ export async function getGithubOrganizations({
);
return false;
}
if (skipOrgNames?.has(org.login.toLowerCase())) {
console.log(`Skipping organization ${org.login} - ignored by user`);
return false;
}
return true;
});
+22
View File
@@ -103,6 +103,28 @@ describe("Scheduler Service - Ignored Repository Handling", () => {
]);
});
test("auto-start gate: enabled=true → should start, enabled=false → should not start even with mirrorInterval", () => {
// Mirror the gate logic from checkAutoStartConfiguration / performInitialAutoStart.
// The enabled flag is the single authoritative signal; a configured
// mirrorInterval is a timing detail and must not bypass a disabled toggle.
const shouldAutoStart = (scheduleConfig?: { enabled?: boolean }) =>
scheduleConfig?.enabled === true;
expect(shouldAutoStart({ enabled: true })).toBe(true);
expect(shouldAutoStart({ enabled: false })).toBe(false);
expect(shouldAutoStart({})).toBe(false);
expect(shouldAutoStart(undefined)).toBe(false);
// Simulating: user disabled scheduling but has a mirrorInterval configured.
// The old code checked `scheduleEnabled || hasMirrorInterval`; the fix
// ensures only the enabled flag is checked.
const configWithIntervalButDisabled = {
scheduleConfig: { enabled: false },
giteaConfig: { mirrorInterval: "8h" },
};
expect(shouldAutoStart(configWithIntervalButDisabled.scheduleConfig)).toBe(false);
});
test("should validate all repository status enum values", () => {
const validStatuses = [
"imported",
+36 -10
View File
@@ -266,6 +266,27 @@ async function runScheduledSync(config: any): Promise<void> {
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
// A `failed` repo whose recorded location still resolves to a
// live same-source mirror (e.g. migrate succeeded but metadata
// failed) must be SYNCED, not re-created — otherwise the
// re-create loop spawns suffixed duplicates (#315). The create
// path also reuses now, but routing to sync here avoids a
// wasted migrate attempt and keeps recovery cheap.
if (repo.status === 'failed' && repository.mirroredLocation) {
const { findExistingMirror } = await import('@/lib/utils/mirror-source-match');
const existing = await findExistingMirror({
repository,
config,
candidateOwner: repository.mirroredLocation.split('/')[0] || '',
candidateName: repository.name,
});
if (existing) {
await syncGiteaRepo({ config, repository });
console.log(`[Scheduler] Re-synced failed repository with live mirror: ${repo.fullName}`);
return;
}
}
await mirrorGithubRepoToGitea({ octokit, repository, config });
console.log(`[Scheduler] Auto-mirrored repository: ${repo.fullName}`);
} catch (error) {
@@ -431,13 +452,16 @@ async function checkAutoStartConfiguration(): Promise<boolean> {
.where(eq(configs.isActive, true));
for (const config of activeConfigs) {
// Check if scheduling is enabled via environment
// Check if scheduling is enabled.
// Note: env-config-loader already sets scheduleConfig.enabled=true when
// GITEA_MIRROR_INTERVAL is set at startup, so the enabled flag is the
// single authoritative gate here. Checking hasMirrorInterval directly
// would allow a configured interval to trigger auto-start even after the
// user explicitly disabled scheduling via the UI.
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// If either SCHEDULE_ENABLED=true or GITEA_MIRROR_INTERVAL is set, we should auto-start
if (scheduleEnabled || hasMirrorInterval) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled}, hasMirrorInterval=${hasMirrorInterval})`);
if (scheduleEnabled) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled})`);
return true;
}
}
@@ -472,10 +496,12 @@ async function performInitialAutoStart(): Promise<void> {
}
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// Only process configs that have scheduling or mirror interval configured
if (!scheduleEnabled && !hasMirrorInterval) {
// Only process configs where scheduling is explicitly enabled.
// env-config-loader already sets enabled=true when GITEA_MIRROR_INTERVAL
// is present, so this single check covers both the UI toggle and the
// env-var boot path without letting a bare interval override a disabled toggle.
if (!scheduleEnabled) {
continue;
}
-303
View File
@@ -1,303 +0,0 @@
/**
* Enhanced handler for starred repositories with improved error handling
*/
import type { Config, Repository } from "./db/schema";
import { Octokit } from "@octokit/rest";
import { processWithRetry } from "./utils/concurrency";
import {
getOrCreateGiteaOrgEnhanced,
getGiteaRepoInfo,
handleExistingNonMirrorRepo,
createOrganizationsSequentially
} from "./gitea-enhanced";
import { mirrorGithubRepoToGitea } from "./gitea";
import { getMirrorStrategyConfig } from "./utils/mirror-strategies";
import { createMirrorJob } from "./helpers";
/**
* Process starred repositories with enhanced error handling
*/
export async function processStarredRepositories({
config,
repositories,
octokit,
}: {
config: Config;
repositories: Repository[];
octokit: Octokit;
}): Promise<void> {
if (!config.userId) {
throw new Error("User ID is required");
}
const strategyConfig = getMirrorStrategyConfig();
console.log(`Processing ${repositories.length} starred repositories`);
console.log(`Using strategy config:`, strategyConfig);
// Step 1: Pre-create organizations to avoid race conditions
if (strategyConfig.sequentialOrgCreation) {
await preCreateOrganizations({ config, repositories });
}
// Step 2: Process repositories with enhanced error handling
await processWithRetry(
repositories,
async (repository) => {
try {
await processStarredRepository({
config,
repository,
octokit,
strategyConfig,
});
return repository;
} catch (error) {
console.error(`Failed to process starred repository ${repository.name}:`, error);
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 2,
retryDelay: 2000,
onProgress: (completed, total, result) => {
const percentComplete = Math.round((completed / total) * 100);
if (result) {
console.log(
`Processed starred repository "${result.name}" (${completed}/${total}, ${percentComplete}%)`
);
}
},
onRetry: (repo, error, attempt) => {
console.log(
`Retrying starred repository ${repo.name} (attempt ${attempt}): ${error.message}`
);
},
}
);
}
/**
* Pre-create all required organizations sequentially
*/
async function preCreateOrganizations({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
// Get unique organization names
const orgNames = new Set<string>();
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
if (starredReposMode === "preserve-owner") {
for (const repo of repositories) {
orgNames.add(repo.organization || repo.owner);
}
} else if (config.githubConfig?.starredReposOrg) {
orgNames.add(config.githubConfig.starredReposOrg);
} else {
orgNames.add("starred");
}
// Add any other organizations based on mirror strategy
for (const repo of repositories) {
if (repo.destinationOrg) {
orgNames.add(repo.destinationOrg);
}
}
console.log(`Pre-creating ${orgNames.size} organizations sequentially`);
// Create organizations sequentially
await createOrganizationsSequentially({
config,
orgNames: Array.from(orgNames),
});
}
/**
* Process a single starred repository with enhanced error handling
*/
async function processStarredRepository({
config,
repository,
octokit,
strategyConfig,
}: {
config: Config;
repository: Repository;
octokit: Octokit;
strategyConfig: ReturnType<typeof getMirrorStrategyConfig>;
}): Promise<void> {
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
// Check if repository exists in Gitea
const existingRepo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (existingRepo) {
if (existingRepo.mirror) {
console.log(`Starred repository ${repository.name} already exists as a mirror`);
// Update database status
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
} else {
// Repository exists but is not a mirror
console.warn(`Starred repository ${repository.name} exists but is not a mirror`);
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo: existingRepo,
strategy: strategyConfig.nonMirrorStrategy,
});
// If we deleted it, continue to create the mirror
if (strategyConfig.nonMirrorStrategy !== "delete") {
return; // Skip if we're not deleting
}
}
}
// Create the mirror
try {
await mirrorGithubRepoToGitea({
octokit,
repository,
config,
});
} catch (error) {
// Enhanced error handling for specific scenarios
if (error instanceof Error) {
const errorMessage = error.message.toLowerCase();
if (errorMessage.includes("already exists")) {
// Handle race condition where repo was created by another process
console.log(`Repository ${repository.name} was created by another process`);
// Check if it's a mirror now
const recheck = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (recheck && recheck.mirror) {
// It's now a mirror, update database
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
}
}
}
throw error;
}
}
/**
* Sync all starred repositories
*/
export async function syncStarredRepositories({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
const strategyConfig = getMirrorStrategyConfig();
console.log(`Syncing ${repositories.length} starred repositories`);
await processWithRetry(
repositories,
async (repository) => {
try {
// Import syncGiteaRepo
const { syncGiteaRepo } = await import("./gitea");
await syncGiteaRepo({
config,
repository,
});
return repository;
} catch (error) {
if (error instanceof Error && error.message.includes("not a mirror")) {
console.warn(`Repository ${repository.name} is not a mirror, handling...`);
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
const repoInfo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (repoInfo) {
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo,
strategy: strategyConfig.nonMirrorStrategy,
});
}
}
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 1,
retryDelay: 1000,
onProgress: (completed, total) => {
const percentComplete = Math.round((completed / total) * 100);
console.log(`Sync progress: ${completed}/${total} (${percentComplete}%)`);
},
}
);
}
+34
View File
@@ -124,3 +124,37 @@ test("githubConfigSchema parses includeCollaboratorRepos with true default", ()
});
expect(parsed.includeCollaboratorRepos).toBe(true);
});
test("skipPersonalRepos defaults to false in githubConfigSchema", () => {
const parsed = githubConfigSchema.parse({
owner: "octo",
type: "personal",
token: "",
});
expect(parsed.skipPersonalRepos).toBe(false);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when true", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: true };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(true);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(true);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when false", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: false };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(false);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(false);
});
test("DB row missing skipPersonalRepos defaults to false on read", () => {
const ui = mapDbToUiConfig({ githubConfig: { owner: "octo", token: "" } });
expect(ui.advancedOptions.skipPersonalRepos).toBe(false);
});
+2
View File
@@ -71,6 +71,7 @@ export function mapUiToDbConfig(
// Advanced options
starredCodeOnly: advancedOptions.starredCodeOnly,
autoMirrorStarred: advancedOptions.autoMirrorStarred ?? false,
skipPersonalRepos: advancedOptions.skipPersonalRepos ?? false,
};
// Map Gitea config to match database schema
@@ -194,6 +195,7 @@ export function mapDbToUiConfig(dbConfig: any): {
// Support both old (skipStarredIssues) and new (starredCodeOnly) field names for backward compatibility
starredCodeOnly: dbConfig.githubConfig?.starredCodeOnly ?? (dbConfig.githubConfig as any)?.skipStarredIssues ?? false,
autoMirrorStarred: dbConfig.githubConfig?.autoMirrorStarred ?? false,
skipPersonalRepos: dbConfig.githubConfig?.skipPersonalRepos ?? false,
};
return {
+420
View File
@@ -0,0 +1,420 @@
import { describe, test, expect } from "bun:test";
import {
normalizeCloneUrl,
cloneUrlsMatch,
isMirrorOfSource,
classifyCandidateName,
findExistingMirror,
} from "./mirror-source-match";
import type { Repository } from "@/lib/db/schema";
import type { Config } from "@/types/config";
// Minimal Repository factory for tests. Only the fields read by the helper
// matter (cloneUrl, mirroredLocation, fullName, name).
function makeRepo(overrides: Partial<Repository> = {}): Repository {
return {
id: "repo-1",
userId: "user-1",
configId: "config-1",
name: "Update",
fullName: "NostalgiaForInfinity/Update",
url: "https://github.com/NostalgiaForInfinity/Update",
cloneUrl: "https://github.com/NostalgiaForInfinity/Update.git",
owner: "NostalgiaForInfinity",
organization: undefined,
mirroredLocation: "",
isPrivate: false,
isForked: false,
forkedFrom: undefined,
hasIssues: false,
isStarred: true,
isArchived: false,
size: 0,
hasLFS: false,
hasSubmodules: false,
language: undefined,
description: undefined,
defaultBranch: "main",
visibility: "public",
status: "imported",
lastMirrored: undefined,
errorMessage: undefined,
createdAt: new Date(),
updatedAt: new Date(),
...overrides,
} as unknown as Repository;
}
const config: Partial<Config> = {
userId: "user-1",
giteaConfig: { url: "https://gitea.example.com", token: "t" } as any,
};
describe("normalizeCloneUrl", () => {
test("strips trailing .git", () => {
expect(normalizeCloneUrl("https://github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips embedded credentials", () => {
expect(normalizeCloneUrl("https://x-access-token:ghp_secret@github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips trailing slash", () => {
expect(normalizeCloneUrl("https://github.com/a/b/")).toBe(
"https://github.com/a/b"
);
});
test("lowercases host (and value)", () => {
expect(normalizeCloneUrl("https://GitHub.com/a/b")).toBe(
"https://github.com/a/b"
);
});
test("returns empty string for blank/invalid input", () => {
expect(normalizeCloneUrl("")).toBe("");
expect(normalizeCloneUrl(null)).toBe("");
expect(normalizeCloneUrl(undefined)).toBe("");
});
test("handles scp-style git URLs via fallback", () => {
expect(normalizeCloneUrl("git@github.com:a/b.git")).toBe("git@github.com:a/b");
});
});
describe("cloneUrlsMatch", () => {
test("https vs token-embedded URL match", () => {
expect(
cloneUrlsMatch(
"https://github.com/a/b.git",
"https://x-access-token:tok@github.com/a/b.git"
)
).toBe(true);
});
test(".git suffix and trailing slash differences match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/a/b.git/")
).toBe(true);
});
test("host case-insensitive match", () => {
expect(
cloneUrlsMatch("https://GITHUB.com/a/b", "https://github.com/a/b")
).toBe(true);
});
test("different repos do not match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/c/d")
).toBe(false);
});
test("empty/unknown URL never matches", () => {
expect(cloneUrlsMatch("", "https://github.com/a/b")).toBe(false);
expect(cloneUrlsMatch("https://github.com/a/b", undefined)).toBe(false);
});
});
describe("isMirrorOfSource", () => {
test("true when mirror with matching original_url", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b.git"
)
).toBe(true);
});
test("false when not a mirror", () => {
expect(
isMirrorOfSource(
{ mirror: false, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url is for a different source (phantom fork)", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/other/repo" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url missing (cannot confirm)", () => {
expect(
isMirrorOfSource({ mirror: true } as any, "https://github.com/a/b")
).toBe(false);
});
test("false for null repoInfo", () => {
expect(isMirrorOfSource(null, "https://github.com/a/b")).toBe(false);
});
});
describe("findExistingMirror", () => {
test("reuses existing same-source mirror at base candidate name (#315)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
expect(match!.repoName).toBe("Update");
});
test("reuses via mirroredLocation even when base name differs (strategy change, #309)", async () => {
// Strategy changed; current candidate name would be "Update" under "starred",
// but the historical mirror lives at "myorg/Update-NostalgiaForInfinity".
const repo = makeRepo({
mirroredLocation: "myorg/Update-NostalgiaForInfinity",
});
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "myorg" && repoName === "Update-NostalgiaForInfinity") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("myorg");
expect(match!.repoName).toBe("Update-NostalgiaForInfinity");
});
test("returns null on genuine different-source collision (regression guard #95/#236)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
// Same name, but it mirrors a DIFFERENT source.
return {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("returns null for phantom fork (non-mirror at the name)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return { mirror: false, original_url: "" } as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("falls back to fresh creation when mirroredLocation is stale (Gitea repo deleted)", async () => {
const repo = makeRepo({ mirroredLocation: "starred/Update" });
// Both the recorded location and the base candidate are gone.
const getRepoInfo = async () => null;
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("matches mirror even when original_url is token-embedded / .git-suffixed", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url:
"https://x-access-token:tok@github.com/NostalgiaForInfinity/Update.git",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
});
test("skips a candidate whose lookup throws and still resolves a later candidate", async () => {
const repo = makeRepo({ mirroredLocation: "myorg/Update" });
const getRepoInfo = async ({ owner }: any) => {
if (owner === "myorg") {
throw new Error("network blip");
}
if (owner === "starred") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
});
});
describe("classifyCandidateName — suffix vs reuse decision (#315/#309)", () => {
const SOURCE = "https://github.com/NostalgiaForInfinity/Update.git";
test("free name → available", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: false,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("available");
});
test("name occupied by OUR same-source mirror → reusable (no suffix, #315)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("reusable");
});
test("name occupied by a DIFFERENT source → taken (suffix, regression #95/#236)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("name occupied by a NON-mirror → taken (phantom-fork guard, #309)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: { mirror: false, original_url: "" } as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("our same-source mirror but DB-claimed by ANOTHER repo → taken (per-user separation)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: true,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("free in Gitea but DB-claimed by another concurrent op → taken", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: true,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("existing mirror but unknown source (no sourceCloneUrl) → taken", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: undefined,
})
).toBe("taken");
});
});
+213
View File
@@ -0,0 +1,213 @@
import type { Config } from "@/types/config";
import type { Repository } from "@/lib/db/schema";
import type { GiteaRepoInfo } from "@/lib/gitea-enhanced";
/**
* Source-identity matching for mirror reuse.
*
* Starred (and other) repos were duplicating on every re-mirror because the
* existence check only asked "does a repo with this name exist?" never
* "is the existing repo a mirror of THIS same GitHub source?". This module
* answers the second question so callers can reuse an existing same-source
* mirror instead of generating a suffixed duplicate. See issues #315 / #309.
*/
/**
* Normalize a git clone URL for source-identity comparison.
* Strips embedded credentials, a trailing ".git", a trailing slash, and
* lowercases the host (hosts are case-insensitive; paths are not). Returns an
* empty string for blank/invalid input so callers can treat it as "unknown".
*/
export function normalizeCloneUrl(rawUrl?: string | null): string {
if (typeof rawUrl !== "string") return "";
let url = rawUrl.trim();
if (!url) return "";
try {
const parsed = new URL(url);
// Drop any embedded credentials (e.g. https://user:token@host/...).
parsed.username = "";
parsed.password = "";
const host = parsed.host.toLowerCase();
// Strip trailing slash(es) first so a ".git/" suffix still normalizes.
const path = parsed.pathname.replace(/\/+$/, "").replace(/\.git$/i, "");
return `${parsed.protocol}//${host}${path}`.toLowerCase();
} catch {
// Fall back to best-effort string normalization for non-standard URLs
// (e.g. scp-style git@host:owner/repo). Strip credentials before "@",
// drop ".git"/trailing slash, and lowercase the whole thing.
url = url.replace(/^([a-z]+:\/\/)[^@/]+@/i, "$1");
url = url.replace(/\/+$/, "").replace(/\.git$/i, "");
return url.toLowerCase();
}
}
/**
* Whether two clone URLs point at the same source repository, ignoring
* credentials, ".git" suffix, trailing slash, and host case.
*/
export function cloneUrlsMatch(a?: string | null, b?: string | null): boolean {
const normA = normalizeCloneUrl(a);
const normB = normalizeCloneUrl(b);
if (!normA || !normB) return false;
return normA === normB;
}
/**
* Whether an existing Gitea repo is a mirror of the given GitHub source.
* Uses Gitea's original_url (the recorded migration source) when present;
* if Gitea didn't expose original_url, we cannot positively confirm the
* source and return false (callers then treat the name as a genuine
* collision rather than risk mapping onto an unrelated repo #309).
*/
export function isMirrorOfSource(
repoInfo: GiteaRepoInfo | null,
sourceCloneUrl?: string | null
): boolean {
if (!repoInfo || !repoInfo.mirror) return false;
return cloneUrlsMatch(repoInfo.original_url, sourceCloneUrl);
}
export type CandidateNameClassification = "available" | "reusable" | "taken";
/**
* Classify a candidate mirror name for the suffix-vs-reuse decision in
* generateUniqueRepoName. Pure (all I/O is pre-resolved by the caller):
* - "available": free in Gitea and not DB-claimed by another repo use it
* - "reusable": occupied in Gitea by a mirror of THIS source, not DB-claimed
* by another repo reuse it (no suffix)
* - "taken": occupied by a different source / non-mirror, or DB-claimed by
* another repo must suffix
*
* A DB claim by a DIFFERENT repo always blocks reuse so two users mirroring the
* same source into a shared org stay separated.
*/
export function classifyCandidateName({
existsInGitea,
claimedByOther,
repoInfo,
sourceCloneUrl,
}: {
existsInGitea: boolean;
claimedByOther: boolean;
repoInfo: GiteaRepoInfo | null;
sourceCloneUrl?: string | null;
}): CandidateNameClassification {
if (existsInGitea) {
if (!claimedByOther && isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return "reusable";
}
return "taken";
}
// Not in Gitea, but possibly claimed in the DB by a concurrent operation.
if (claimedByOther) return "taken";
return "available";
}
export interface ExistingMirrorMatch {
owner: string;
repoName: string;
repoInfo: GiteaRepoInfo;
}
/**
* Resolve an existing same-source mirror for a repository, if one exists.
*
* Resolution order (backward compatible):
* 1. The recorded repository.mirroredLocation if it still resolves to a
* live mirror of THIS source, reuse it even when the base candidate name
* differs from the current naming strategy (handles strategy changes #309).
* 2. The provided candidate owner/name if that resolves to a live mirror of
* THIS source, reuse it (handles the self-collision that drove suffixing #315).
*
* Returns null when no live same-source mirror is found (caller should create
* a fresh mirror, generating a unique name if the candidate name is taken by a
* DIFFERENT source).
*/
export async function findExistingMirror({
repository,
config,
candidateOwner,
candidateName,
getRepoInfo,
}: {
repository: Repository;
config: Partial<Config>;
candidateOwner: string;
candidateName: string;
// Injectable for testing; defaults to the real Gitea lookup.
getRepoInfo?: (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => Promise<GiteaRepoInfo | null>;
}): Promise<ExistingMirrorMatch | null> {
const lookup =
getRepoInfo ??
(async (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => {
const { getGiteaRepoInfo } = await import("@/lib/gitea-enhanced");
return getGiteaRepoInfo(args);
});
const sourceCloneUrl = repository.cloneUrl;
// Candidate locations to probe, in priority order. Dedupe so we don't issue
// the same HTTP lookup twice when mirroredLocation equals the candidate.
const candidates: Array<{ owner: string; repoName: string }> = [];
const seen = new Set<string>();
const pushCandidate = (owner?: string | null, repoName?: string | null) => {
const o = (owner || "").trim();
const r = (repoName || "").trim();
if (!o || !r) return;
const key = `${o}/${r}`.toLowerCase();
if (seen.has(key)) return;
seen.add(key);
candidates.push({ owner: o, repoName: r });
};
if (repository.mirroredLocation && repository.mirroredLocation.trim()) {
const slashIndex = repository.mirroredLocation.indexOf("/");
if (slashIndex > 0 && slashIndex < repository.mirroredLocation.length - 1) {
pushCandidate(
repository.mirroredLocation.slice(0, slashIndex),
repository.mirroredLocation.slice(slashIndex + 1)
);
}
}
pushCandidate(candidateOwner, candidateName);
for (const candidate of candidates) {
let repoInfo: GiteaRepoInfo | null;
try {
repoInfo = await lookup({
config,
owner: candidate.owner,
repoName: candidate.repoName,
});
} catch (error) {
// A failed lookup (network/auth) should not be mistaken for "no mirror";
// skip this candidate and let the caller fall back to its normal flow.
console.warn(
`[Mirror] Could not look up ${candidate.owner}/${candidate.repoName} while resolving existing mirror for ${repository.fullName}: ${
error instanceof Error ? error.message : String(error)
}`
);
continue;
}
if (isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return {
owner: candidate.owner,
repoName: candidate.repoName,
repoInfo: repoInfo as GiteaRepoInfo,
};
}
}
return null;
}
+5 -5
View File
@@ -47,8 +47,11 @@ export async function POST(context: APIContext) {
}
try {
// Use Better Auth server API to register OAuth2 application
const response = await auth.api.registerOAuthApplication({
// Use Better Auth server API to register OAuth2 client (RFC 7591
// dynamic client registration via @better-auth/oauth-provider).
// Note: jwks / jwks_uri / metadata are not part of the new
// registration body and are intentionally omitted.
const response = await auth.api.registerOAuthClient({
body: {
client_name,
redirect_uris,
@@ -61,9 +64,6 @@ export async function POST(context: APIContext) {
contacts,
tos_uri,
policy_uri,
jwks_uri,
jwks,
metadata,
software_id,
software_version,
software_statement,
+12 -15
View File
@@ -152,25 +152,14 @@ export async function POST(context: APIContext) {
headers.set("cookie", cookieHeader);
}
// Register the SSO provider using Better Auth's API
const response = await auth.api.registerSSOProvider({
// Register the SSO provider using Better Auth's API.
// auth.api.* returns the parsed result directly (not a fetch Response); it
// throws APIError on failure, which createSecureErrorResponse handles below.
const result = await auth.api.registerSSOProvider({
body: registrationBody,
headers,
});
if (!response.ok) {
const error = await response.text();
return new Response(
JSON.stringify({ error: `Failed to register SSO provider: ${error}` }),
{
status: response.status,
headers: { "Content-Type": "application/json" },
}
);
}
const result = await response.json();
// Mirror provider entry into local SSO table for UI listing
try {
const existing = await db
@@ -183,6 +172,13 @@ export async function POST(context: APIContext) {
issuer: registrationBody.issuer,
domain: registrationBody.domain,
organizationId: registrationBody.organizationId,
// Mark this provider as trusted for the `domain` it was registered
// under. Better Auth's SSO plugin gates account auto-linking on this
// flag together with an email-domain match (validateEmailDomain), so
// sign-ins from users outside the registered domain are NOT
// auto-linked even though the provider is trusted. The plugin's own
// create call hardcodes this to false, so we set it here.
domainVerified: true,
updatedAt: new Date(),
};
@@ -210,6 +206,7 @@ export async function POST(context: APIContext) {
userId: user.id,
providerId: registrationBody.providerId,
organizationId: registrationBody.organizationId,
domainVerified: true,
});
}
} catch (mirroringError) {
+89
View File
@@ -0,0 +1,89 @@
import type { APIRoute } from "astro";
import { db, repositories } from "@/lib/db";
import { and, eq, inArray } from "drizzle-orm";
import { repoStatusEnum } from "@/types/Repository";
import { createMirrorJob } from "@/lib/helpers";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
/**
* POST /api/job/cancel-pending
*
* Sets this user's repositories that are waiting to be mirrored
* (status: "imported" or "failed") to "ignored", preventing the scheduler
* from picking them up. Repos with status "mirroring" or "syncing" are
* left alone because they have in-flight work that cannot be aborted here.
*
* Returns the count of affected repositories and logs one activity entry.
*/
export const POST: APIRoute = async ({ request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
// Statuses that represent queued-but-not-started work.
// "imported" → repo was discovered, never mirrored
// "failed" → last mirror attempt failed; scheduler will retry
const cancelableStatuses = ["imported", "failed"] as const;
// Fetch repos to cancel so we can count them and log meaningful details.
const toCancel = await db
.select({ id: repositories.id })
.from(repositories)
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.status, cancelableStatuses),
),
);
const cancelCount = toCancel.length;
if (cancelCount > 0) {
const ids = toCancel.map((r) => r.id);
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("ignored"),
updatedAt: new Date(),
errorMessage: "Cancelled by user — set to ignored via Stop Pending Mirrors.",
})
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.id, ids),
),
);
}
// Log a single activity summarising the bulk action.
await createMirrorJob({
userId,
message: `Stopped pending mirrors: ${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} set to Ignored`,
details:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} with status "imported" or "failed" have been set to "ignored". ` +
`They can be re-enabled from the Repositories page.`
: `No repositories in a pending state were found for this user.`,
status: cancelCount > 0 ? "ignored" : "skipped",
skipDuplicateEvent: false,
skipNotification: true,
});
return new Response(
JSON.stringify({
success: true,
message:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y has" : "ies have"} been set to Ignored.`
: "No repositories in a pending state were found.",
cancelledCount: cancelCount,
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
} catch (error) {
return createSecureErrorResponse(error, "cancel pending mirrors", 500);
}
};
+1 -51
View File
@@ -1,5 +1,5 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { db, repositories } from "@/lib/db";
import { eq, and } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
@@ -62,53 +62,3 @@ export const PATCH: APIRoute = async (context) => {
}
};
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const repoId = context.params.id;
if (!repoId) {
return new Response(JSON.stringify({ error: "Repository ID is required" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
const [existingRepo] = await db
.select()
.from(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)))
.limit(1);
if (!existingRepo) {
return new Response(
JSON.stringify({ error: "Repository not found" }),
{
status: 404,
headers: { "Content-Type": "application/json" },
}
);
}
await db
.delete(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)));
await db
.delete(mirrorJobs)
.where(and(eq(mirrorJobs.repositoryId, repoId), eq(mirrorJobs.userId, userId)));
return new Response(
JSON.stringify({ success: true }),
{
status: 200,
headers: { "Content-Type": "application/json" },
}
);
} catch (error) {
return createSecureErrorResponse(error, "Delete repository", 500);
}
};
+49
View File
@@ -0,0 +1,49 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { eq, and, inArray } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const body = await context.request.json();
const { ids } = body;
if (!Array.isArray(ids) || ids.length === 0) {
return new Response(JSON.stringify({ error: "ids must be a non-empty array" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
// Verify all repos belong to this user before deleting
const owned = await db
.select({ id: repositories.id })
.from(repositories)
.where(and(inArray(repositories.id, ids), eq(repositories.userId, userId)));
const ownedIds = owned.map((r) => r.id);
if (ownedIds.length === 0) {
return new Response(JSON.stringify({ error: "No matching repositories found" }), {
status: 404,
headers: { "Content-Type": "application/json" },
});
}
await db.transaction(async (tx) => {
await tx.delete(mirrorJobs).where(and(inArray(mirrorJobs.repositoryId, ownedIds), eq(mirrorJobs.userId, userId)));
await tx.delete(repositories).where(and(inArray(repositories.id, ownedIds), eq(repositories.userId, userId)));
});
return new Response(
JSON.stringify({ success: true, deleted: ownedIds.length }),
{ status: 200, headers: { "Content-Type": "application/json" } }
);
} catch (error) {
return createSecureErrorResponse(error, "Bulk delete repositories", 500);
}
};
+109 -74
View File
@@ -1,26 +1,61 @@
import type { APIContext } from "astro";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
import { db, oauthApplications } from "@/lib/db";
import { nanoid } from "nanoid";
import { auth } from "@/lib/auth";
import { db, oauthClients } from "@/lib/db";
import { eq } from "drizzle-orm";
import { generateRandomString } from "@/lib/utils";
// GET /api/sso/applications - List all OAuth applications
// Backward-compatible OAuth application management API.
//
// Migrated from the deprecated `oidc-provider` plugin to
// `@better-auth/oauth-provider`. Clients now live in the `oauth_clients`
// table and are managed by the plugin (which generates and hashes the
// client secret), so mutations delegate to `auth.api.*OAuthClient`. Reads
// are served straight from the table and mapped back to the legacy response
// shape (`redirectURLs` as a comma-separated string) so existing consumers
// — notably the consent page — keep working.
// `redirectUris` is stored by the adapter as a JSON-encoded string[].
function parseRedirectUris(value: unknown): string[] {
if (Array.isArray(value)) return value as string[];
if (typeof value === "string" && value.length > 0) {
try {
const parsed = JSON.parse(value);
if (Array.isArray(parsed)) return parsed;
} catch {
// Legacy comma-separated fallback
return value.split(",").map((s) => s.trim()).filter(Boolean);
}
}
return [];
}
// Map a new oauth_clients row onto the legacy application response shape.
function toLegacyApplication(row: typeof oauthClients.$inferSelect) {
return {
id: row.id,
clientId: row.clientId,
name: row.name ?? "",
redirectURLs: parseRedirectUris(row.redirectUris).join(","),
type: row.type ?? "web",
disabled: row.disabled ?? false,
createdAt: row.createdAt,
updatedAt: row.updatedAt,
// Never expose the (hashed) client secret in list responses
clientSecret: undefined,
};
}
// GET /api/sso/applications - List all OAuth clients
export async function GET(context: APIContext) {
try {
const { user, response } = await requireAuth(context);
const { response } = await requireAuth(context);
if (response) return response;
const applications = await db.select().from(oauthApplications);
const rows = await db.select().from(oauthClients);
const sanitized = rows.map(toLegacyApplication);
// Don't send client secrets in list response
const sanitizedApps = applications.map(app => ({
...app,
clientSecret: undefined,
}));
return new Response(JSON.stringify(sanitizedApps), {
return new Response(JSON.stringify(sanitized), {
status: 200,
headers: { "Content-Type": "application/json" },
});
@@ -29,10 +64,10 @@ export async function GET(context: APIContext) {
}
}
// POST /api/sso/applications - Create a new OAuth application
// POST /api/sso/applications - Register a new OAuth client
export async function POST(context: APIContext) {
try {
const { user, response } = await requireAuth(context);
const { response } = await requireAuth(context);
if (response) return response;
const body = await context.request.json();
@@ -49,27 +84,27 @@ export async function POST(context: APIContext) {
);
}
// Generate client credentials
const clientId = `client_${generateRandomString(32)}`;
const clientSecret = `secret_${generateRandomString(48)}`;
const redirect_uris = Array.isArray(redirectURLs)
? redirectURLs
: String(redirectURLs)
.split(",")
.map((s) => s.trim())
.filter(Boolean);
// Insert new application
const [newApp] = await db
.insert(oauthApplications)
.values({
id: nanoid(),
clientId,
clientSecret,
name,
redirectURLs: Array.isArray(redirectURLs) ? redirectURLs.join(",") : redirectURLs,
// Delegate to the OAuth provider plugin so the client_id / client_secret
// are generated and the secret is stored using the plugin's hashing.
const created = await auth.api.createOAuthClient({
headers: context.request.headers,
body: {
client_name: name,
redirect_uris,
type,
metadata: metadata ? JSON.stringify(metadata) : null,
userId: user.id,
disabled: false,
})
.returning();
},
});
return new Response(JSON.stringify(newApp), {
// The plugin returns RFC-style snake_case fields. Surface them plus the
// one-time client_secret (only returned here, never again).
return new Response(JSON.stringify(created), {
status: 201,
headers: { "Content-Type": "application/json" },
});
@@ -78,18 +113,19 @@ export async function POST(context: APIContext) {
}
}
// PUT /api/sso/applications/:id - Update an OAuth application
// PUT /api/sso/applications?id=<clientId> - Update an OAuth client
export async function PUT(context: APIContext) {
try {
const { user, response } = await requireAuth(context);
const { response } = await requireAuth(context);
if (response) return response;
const url = new URL(context.request.url);
const appId = url.pathname.split("/").pop();
// Accept the OAuth client_id either from the query string or the path.
const clientId = url.searchParams.get("id") || url.pathname.split("/").pop();
if (!appId) {
if (!clientId) {
return new Response(
JSON.stringify({ error: "Application ID is required" }),
JSON.stringify({ error: "Client ID is required" }),
{
status: 400,
headers: { "Content-Type": "application/json" },
@@ -98,35 +134,26 @@ export async function PUT(context: APIContext) {
}
const body = await context.request.json();
const { name, redirectURLs, disabled, metadata } = body;
const { name, redirectURLs, disabled } = body;
const updateData: any = {};
if (name !== undefined) updateData.name = name;
const updateBody: Record<string, unknown> = { client_id: clientId };
if (name !== undefined) updateBody.client_name = name;
if (disabled !== undefined) updateBody.disabled = disabled;
if (redirectURLs !== undefined) {
updateData.redirectURLs = Array.isArray(redirectURLs)
? redirectURLs.join(",")
: redirectURLs;
}
if (disabled !== undefined) updateData.disabled = disabled;
if (metadata !== undefined) updateData.metadata = JSON.stringify(metadata);
const [updated] = await db
.update(oauthApplications)
.set({
...updateData,
updatedAt: new Date(),
})
.where(eq(oauthApplications.id, appId))
.returning();
if (!updated) {
return new Response(JSON.stringify({ error: "Application not found" }), {
status: 404,
headers: { "Content-Type": "application/json" },
});
updateBody.redirect_uris = Array.isArray(redirectURLs)
? redirectURLs
: String(redirectURLs)
.split(",")
.map((s) => s.trim())
.filter(Boolean);
}
return new Response(JSON.stringify({ ...updated, clientSecret: undefined }), {
const updated = await auth.api.updateOAuthClient({
headers: context.request.headers,
body: updateBody as any,
});
return new Response(JSON.stringify(updated), {
status: 200,
headers: { "Content-Type": "application/json" },
});
@@ -135,18 +162,18 @@ export async function PUT(context: APIContext) {
}
}
// DELETE /api/sso/applications/:id - Delete an OAuth application
// DELETE /api/sso/applications?id=<clientId> - Delete an OAuth client
export async function DELETE(context: APIContext) {
try {
const { user, response } = await requireAuth(context);
const { response } = await requireAuth(context);
if (response) return response;
const url = new URL(context.request.url);
const appId = url.searchParams.get("id");
const clientId = url.searchParams.get("id");
if (!appId) {
if (!clientId) {
return new Response(
JSON.stringify({ error: "Application ID is required" }),
JSON.stringify({ error: "Client ID is required" }),
{
status: 400,
headers: { "Content-Type": "application/json" },
@@ -154,18 +181,26 @@ export async function DELETE(context: APIContext) {
);
}
const deleted = await db
.delete(oauthApplications)
.where(eq(oauthApplications.id, appId))
.returning();
// Ensure the client exists so we can return a 404 (the plugin endpoint
// may otherwise succeed silently).
const existing = await db
.select()
.from(oauthClients)
.where(eq(oauthClients.clientId, clientId))
.limit(1);
if (deleted.length === 0) {
if (existing.length === 0) {
return new Response(JSON.stringify({ error: "Application not found" }), {
status: 404,
headers: { "Content-Type": "application/json" },
});
}
await auth.api.deleteOAuthClient({
headers: context.request.headers,
body: { client_id: clientId },
});
return new Response(JSON.stringify({ success: true }), {
status: 200,
headers: { "Content-Type": "application/json" },
@@ -173,4 +208,4 @@ export async function DELETE(context: APIContext) {
} catch (error) {
return createSecureErrorResponse(error, "SSO applications API");
}
}
}
+12 -3
View File
@@ -49,13 +49,20 @@ export const POST: APIRoute = async ({ request, locals }) => {
const githubUsername = config.githubConfig?.owner || undefined;
const octokit = createGitHubClient(decryptedToken, userId, githubUsername);
// Load ignored orgs from the DB so we can skip them during import
const ignoredOrgRows = await db
.select({ normalizedName: organizations.normalizedName })
.from(organizations)
.where(and(eq(organizations.userId, userId), eq(organizations.status, "ignored")));
const ignoredOrgNames = new Set(ignoredOrgRows.map((o) => o.normalizedName));
// Fetch GitHub data in parallel
const [basicAndForkedRepos, starredRepos, orgResult] = await Promise.all([
getGithubRepositories({ octokit, config }),
config.githubConfig?.includeStarred
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
getGithubOrganizations({ octokit, config }),
getGithubOrganizations({ octokit, config, skipOrgNames: ignoredOrgNames }),
]);
const { organizations: gitOrgs, failedOrgs } = orgResult;
@@ -152,7 +159,9 @@ export const POST: APIRoute = async ({ request, locals }) => {
const existingOrgMap = new Map(existingOrgs.map((o) => [o.normalizedName, o.status]));
insertedRepos = newRepos.filter(
(r) => !existingRepoNames.has(r.normalizedFullName)
(r) =>
!existingRepoNames.has(r.normalizedFullName) &&
(!r.organization || !ignoredOrgNames.has(r.organization.toLowerCase()))
);
insertedOrgs = newOrgs.filter((o) => !existingOrgMap.has(o.normalizedName));
@@ -258,7 +267,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
newRepositories: insertedRepos.length,
newOrganizations: insertedOrgs.length,
skippedDisabledRepositories: allGithubRepos.length - mirrorableGithubRepos.length,
failedOrgs: failedOrgs.map((o) => o.name),
failedOrgs: failedOrgs.filter((o) => !ignoredOrgNames.has(o.name.toLowerCase())).map((o) => o.name),
recoveredOrgs: recoveredOrgCount,
},
});
+1
View File
@@ -86,6 +86,7 @@ export interface AdvancedOptions {
skipForks: boolean;
starredCodeOnly: boolean;
autoMirrorStarred?: boolean;
skipPersonalRepos?: boolean;
}
export interface SaveConfigApiRequest {