Compare commits

...

24 Commits

Author SHA1 Message Date
Arunavo Ray 06bfb49e0e chore: bump version to 3.20.4 2026-07-02 15:46:20 +05:30
ARUNAVO RAY b5e0c58708 fix: stop false-positive orphan archiving and heal sync 405s on archived-* renamed mirrors (#331) (#336)
Three related fixes for the "repos keep getting archived and then fail to
sync with HTTP 405" report:

1. Orphan cleanup no longer archives on bulk-list absence alone.
   Repos added via the "+" Add Repository dialog (foreign owner, not
   starred) can never appear in the authenticated bulk fetches, so every
   cleanup cycle deterministically flagged them as orphaned and archived
   them. identifyOrphanedRepositories() now runs a targeted per-repo
   confirmation (starred check or repos.get) and only treats a clean 404
   as gone; any other outcome fails safe.

2. The archived-* rename is persisted. archiveGiteaRepo() now returns
   the actual post-rename name and the cleanup service records it in
   mirroredLocation, so the DB no longer points at a name that only
   301-redirects.

3. Sync self-heals repos renamed in Gitea/Forgejo. Requests to a
   renamed repo get a 301; fetch follows it, downgrading POST to GET,
   which lands on the POST-only mirror-sync endpoint as a 405. The sync
   candidate loop now adopts the canonical owner/name from the GET
   response body before POSTing, tries an archived-{name} fallback for
   archived repos (guarded by an original_url source match), and keeps
   archived repos archived: no mirror-interval PATCH, status stays
   'archived' per the documented Manual Sync contract.

Also hardens mirrorGitHubReleasesToGitea to derive GitHub coordinates
from fullName so Gitea-side names can never leak into GitHub API calls.

Verified end-to-end on Forgejo 15.0.3 (rootless): pre-fix reproduces the
exact 405; post-fix the stale-name sync succeeds (GET stale -> 301, GET
canonical -> 200, POST canonical mirror-sync -> 200), archived repos keep
interval "0s" with no PATCH issued, and non-archived renamed repos heal
and get the configured interval applied.
2026-07-02 15:46:00 +05:30
Arunavo Ray 74606f0a5f chore: bump version to 3.20.3 2026-07-01 08:13:15 +05:30
ARUNAVO RAY 187ecc5d60 fix: correctly mirror Gitea release titles and issue/PR labels (#334 + sibling) (#335)
* fix(releases): send Gitea release title as `name`, not `title` (#334)

Gitea/Forgejo expose the release title through the JSON field `name`
(the API Go struct is `Title string `+"`"+`json:"name"`+"`"+`). The release
create and update payloads sent `title:` instead, which Gitea silently
ignores, so every mirrored release landed with a blank title.

Verified live against Gitea 1.24.7: a POST/PATCH with `title` yields
`name: ""`; the same call with `name` sets the title correctly. The
update path also self-heals previously-mirrored releases whose names were
left blank, since the existing-vs-expected name comparison already drives
a PATCH.

Adds gita-release-name.test.ts, which drives the real
mirrorGitHubReleasesToGitea create/update paths with a mocked fetch and
asserts the payload carries `name` (and never `title`).

* fix(issues): reconcile labels on issue/PR update via the labels sub-resource (#334 sibling)

Gitea/Forgejo's `EditIssueOption` has no `labels` field (only
`CreateIssueOption` does), so a `labels` key in a `PATCH .../issues/{index}`
body is silently dropped — the same silent-ignore class as the release
`title` vs `name` bug. The issue and PR-as-issue update paths sent `labels`
in the PATCH body, so label changes never propagated onto already-mirrored
issues (and a deadlock-orphaned issue recovered via PATCH never got its
labels).

Fix: add `reconcileGiteaIssueLabels`, which replaces the label set via
`PUT .../issues/{index}/labels` (idempotent — adds new, removes deleted).
Call it on the two issue update paths and the two PR-issue update paths,
and drop the dead `labels` key from those PATCH bodies. Labels on freshly
created issues still come from CreateIssueOption on the POST.

Verified live against Gitea 1.24.7 (PATCH ignores labels; PUT applies them)
and end-to-end (a drifted mirrored issue reconciled from no-labels to its
GitHub label set). Adds gitea-issue-labels.test.ts driving the real
mirrorGitRepoIssuesToGitea update path; the test carries a self-contained
http-client mock so it is immune to another suite's global module mock.

* test: make #334 regression tests deterministic via pure payload builders

The prior tests drove the real mirror functions with a global `fetch` mock.
That is order/version-fragile: another suite installs a process-global
`mock.module("@/lib/http-client")`, and bun 1.3.13 (CI) runs test files
concurrently, so `globalThis.fetch` races across files and
`isRepoPresentInGitea` (raw fetch) intermittently sees the wrong mock —
green locally on bun 1.3.6, red in CI.

Extract the payload construction into pure, exported builders and assert on
those instead (the repo's existing `classify*` pattern): buildGiteaReleasePayload
(create+update send `name`, never `title`), buildGiteaIssueEditPayload (edit
body never carries `labels`), buildGiteaIssueLabelsPayload (labels sub-resource
body). Behavior is unchanged — the builders return the exact same objects the
call sites built inline — and the fixes remain verified live on Gitea 1.24.7.
2026-07-01 08:12:36 +05:30
Arunavo Ray 632bbd0d4a chore: bump version to 3.20.2 2026-06-24 17:19:10 +05:30
ARUNAVO RAY 0b65e40784 fix(releases): create releases only for tags present in Gitea; stop sending target (#331) (#333)
Release creation failed on some Gitea/Forgejo instances with
"HTTP 404: The target couldn't be found", so no release (and therefore no
assets) was ever created — re-syncing never recovered.

Root cause: the create payload always sent `target: target_commitish`
(e.g. "main"). When the release's git tag is not yet present in the Gitea
mirror — which happens when Gitea's own git mirror clone lags behind the
metadata sync — Gitea tries to *create* the tag from `target`; if that ref
can't be resolved it returns a generic 404 ("The target couldn't be
found"), and if it can, it would create a brand-new tag at the wrong commit.

Reproduced the reporter's exact stack (Forgejo 15 rootless + read_only +
cap_drop ALL + postgres, plus Gitea 1.20-1.26 and Forgejo 1.21-15): a
healthy repo always succeeds — the 404 only occurs when the tag is absent
at create time.

Fix:
- Before creating a release, verify the git tag already exists in Gitea.
  If it isn't synced yet, skip it (logged) and let a later sync create it
  once the mirror has the tag — never create a tag via `target`.
- Drop the `target` field from both the create and update payloads. For a
  mirror the tag is synced from upstream, so Gitea attaches the release to
  the existing tag; `target` is unnecessary and is what triggers the 404.
- Surface skipped-missing-tag releases in the summary log for diagnosability.

Verified end-to-end against the real mirror function on a Forgejo instance:
a release whose tag exists is created with its assets; a release whose tag
was removed is skipped cleanly (no 404, no bogus tag) and picked up once the
tag is present.
2026-06-24 17:18:46 +05:30
Arunavo Ray 4a8b4f6ff3 chore: bump version to 3.20.1 2026-06-23 23:07:16 +05:30
ARUNAVO RAY 1d9dfdeb70 fix(releases): mirror assets idempotently so missing assets self-heal (#331) (#332)
Release assets were uploaded only on the create path of
mirrorGitHubReleasesToGitea(). When a Gitea release already existed, the
update path PATCHed the changelog/title and `continue`d without ever
touching assets. So any release whose assets were not fully uploaded on
that single create-path run — first sync interrupted, a transient
download/upload failure, large multi-MB assets, etc. — stayed permanently
asset-less, and re-syncing always hit the update path and could never
recover it. Asset failures were also swallowed to console.error, so the
job still reported success (the "no errors in the logs" in #331).

Reproduced on a real Forgejo pull-mirror with shauninman/MinUI: a GitHub
release with two ~35-40MB binaries became a Gitea release with 0 assets
(just Forgejo's auto-generated source archive), and re-syncing left it at
0 while logging "Updating existing release".

Fix:
- Add reconcileReleaseAssets(), an idempotent reconciler run on BOTH the
  create and update paths. It compares Gitea's existing attachments to
  GitHub's by name+size: skips matches, uploads missing ones, replaces
  size-mismatched copies. Existing broken releases self-heal on next sync.
- Extract the pure decision into classifyAssetsForReconciliation() for
  unit testing (the absence of asset tests is why this slipped past #310).
- Surface asset upload counts in the summary and emit a visible warning
  when any fail, instead of silently swallowing them.
- Add 5 unit tests covering the broken state, partial backfill,
  idempotency, size-mismatch replacement, and the no-assets case.

Verified end-to-end against the real function on a Forgejo pull-mirror:
0 -> 2 assets backfilled, already-present assets skipped (no re-download),
second run uploads 0.
2026-06-23 23:06:43 +05:30
Arunavo Ray 079df29f44 chore: bump version to 3.20.0 2026-06-19 08:44:03 +05:30
ARUNAVO RAY dff3cafb5e fix(config): persist Name Collision Strategy (starredDuplicateStrategy) (#326) (#328)
The "Name collision strategy" dropdown (starredDuplicateStrategy) never
persisted: the field was absent from both directions of the UI<->DB config
mapper. On save, mapUiToDbConfig dropped it before the DB write; on load,
mapDbToUiConfig never read it, so the UI reset to the "suffix" (repo-owner)
default. Mirror logic in gitea.ts then read undefined and also defaulted to
suffix — so repos really were created with that pattern regardless of the
user's choice. It has been broken since the field was introduced.

- Map starredDuplicateStrategy in mapUiToDbConfig and mapDbToUiConfig
- Add STARRED_DUPLICATE_STRATEGY env var for parity (reporter could not
  work around it via compose because no env var existed) + docs
- Round-trip tests covering save, load, and the missing-field default
2026-06-19 08:43:27 +05:30
ARUNAVO RAY 6ca7c0eec0 feat(github): add organization allowlist to mirror only selected orgs (#327)
Repository discovery requested the `organization_member` affiliation
unconditionally, so repos from every org a user belongs to were imported —
even orgs they never explicitly added. `skipPersonalRepos` only dropped
user-owned repos and left org repos unfiltered, which surprised users who
expected "only mirror org repos" to mean "only the orgs I chose" (reported
on #304).

Wire up the previously-dormant `includeOrganizations` config field as an
opt-in allowlist: when non-empty, only repos owned by the listed
organizations are imported. Empty = all org repos (backward-compatible).
Owned and collaborator repos are never restricted, so it composes cleanly
with `skipPersonalRepos`.

- Filter org repos by the allowlist in getGithubRepositories
- Add includeAllOrgsOverride so the cleanup service bypasses the allowlist
  and never false-orphans a previously-mirrored repo from an org the user
  later removes from the list
- UI control under Filtering & Behavior; INCLUDE_ORGANIZATIONS env var
- Case-insensitive dedup/trim in the UI<->DB mapper round-trip
- 7 unit tests covering the filter, composition, and the cleanup override
2026-06-19 08:42:17 +05:30
ARUNAVO RAY 6ebf1916e8 chore(deps): update dependencies across app and website (#325)
* chore(deps): update app dependencies to latest in-range versions

* chore(deps): update website (www) dependencies to latest in-range versions
2026-06-14 12:25:45 +05:30
Arunavo Ray 91de0d1030 chore: bump version to 3.19.1 2026-06-14 12:07:48 +05:30
Brendan Davidson 85bd1f4042 Repository table bulk actions (#322)
* Handle indexing when shift + clicking in the repository table

* Move the buttons when selecting rows

* Add in a bulk delete func in the repositories table

* Add bulk delete handler

* Make the single action use the bulk delete

* Delete the single repository id handler
2026-06-14 10:14:51 +05:30
Brendan Davidson 4a28015685 Skip the user defined orgs to ignore (#323) 2026-06-14 10:14:48 +05:30
Arunavo Ray da23941369 chore: bump version to 3.19.0 2026-06-13 09:14:33 +05:30
Brendan Davidson 906ce57e8c Handle indexing when shift + clicking in the repository table (#316) 2026-06-13 09:14:02 +05:30
Arunavo Ray 1b84c75a97 chore: bump version to 3.18.0 2026-06-13 08:17:20 +05:30
ARUNAVO RAY 0b6b6b76bf feat(github): add skipPersonalRepos toggle to mirror only org repos (#304) (#320)
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
2026-06-13 08:00:50 +05:30
ARUNAVO RAY 7610a614da fix: scheduler auto-start gate, backup clone URL, cancel-pending action, actionable 405 (#319)
* fix(scheduler): make enabled flag authoritative for auto-start

checkAutoStartConfiguration() and performInitialAutoStart() previously
used `scheduleEnabled || hasMirrorInterval`, allowing a configured
GITEA_MIRROR_INTERVAL to trigger boot-time auto-start even after the
user disabled scheduling via the UI toggle.

env-config-loader already writes scheduleConfig.enabled=true when
GITEA_MIRROR_INTERVAL is set at container startup, so the interval is
a timing detail, not an enable signal. The documented env-var contract
is preserved: GITEA_MIRROR_INTERVAL at boot → env-config-loader sets
enabled=true → auto-start fires. But a later UI disable now sticks.

Add a focused unit test for the gate logic.

* fix(backup): always derive clone URL from user-configured Gitea URL

The pre-sync backup preferred repoInfo.clone_url, which reflects
Gitea's ROOT_URL setting. In Tailscale MagicDNS deployments (and any
setup where ROOT_URL is an external address), this URL is unreachable
from the app itself, causing bundle backup to fail.

Always build the clone URL as:
  ${config.giteaConfig.url.trimEnd('/')}/${owner}/${repo}.git

This matches the URL the app already uses for all other Gitea API
calls and is guaranteed reachable.

* feat(jobs): cancel-pending endpoint + fix misleading Delete All copy

Add POST /api/job/cancel-pending that sets the current user's
repositories with status "imported" or "failed" to "ignored",
preventing the scheduler from re-queuing them. In-flight "mirroring"
rows are left alone. Returns the count and logs one activity entry.

Fix the "Delete All Activities" dialog to clearly state it only clears
the history log and does not stop pending work. Rename button/title to
"Clear History" so intent is unambiguous.

Add a "Stop Pending Mirrors" button (StopCircle icon, amber) in both
mobile and desktop activity log toolbars, with a confirmation dialog
explaining repos are set to Ignored and can be re-enabled from the
Repositories page.

* fix(sync): actionable 405 error for non-pull-mirror repos

Gitea returns HTTP 405 with an empty body when the target repository is
no longer a pull mirror — e.g. the mirror was auto-disabled by Gitea or
the repository lost its mirror state after a manual edit.

Previously this fell through to the generic error handler which stored
the raw HttpError message (often empty) giving the user no guidance.

Now a 405 response is caught alongside the existing 400 handler and
sets the repository to "failed" with an actionable error message:

  "Gitea reports this repository is not a pull mirror (HTTP 405).
  In Gitea check Settings → Mirror Settings; if the mirror section is
  missing, delete the repository in Gitea and re-mirror it from
  gitea-mirror."

The same message is written to the activity log for visibility in the
dashboard.
2026-06-13 08:00:47 +05:30
ARUNAVO RAY c28dcc209f fix(releases): stop delete/recreate cycle on permanent order mismatch (#310) (#318)
Root cause (Theory A): the `needsRecreation` check compared GitHub
published_at-based expected indices against Gitea's API order. Gitea mirror
repos sort releases by tag-commit date, which can permanently disagree with
published_at order (e.g. unaconfig_dart v0.1.0 published after v0.1.1 but
tagged before). This made `currentExpectedIdx < nextExpectedIdx` evaluate
true on every sync, triggering delete-all-and-recreate forever — spamming
Gitea's activity feed with "released X" events (#310).

Fix: replace the destructive order-check machinery with set-based
reconciliation via `classifyReleasesForReconciliation`. Releases are
created when missing in Gitea and skipped (or PATCH-updated if content
drifted) when already present. No deletions are ever triggered by ordering.
Retain the existing release-limit trimming (retention cleanup) unchanged.

Also removes the 1-second per-release delay that was only needed for the
creation-order dance, significantly speeding up initial mirrors.

Adds unit tests covering: normal ordered repos, the unaconfig_dart inversion
fixture, missing→create, present→skip, and edge cases.
2026-06-13 08:00:44 +05:30
ARUNAVO RAY 40ee3cbc44 fix(mirror): reuse existing same-source mirrors instead of creating suffixed duplicates (#315) (#317)
Starred (and other) repos duplicated on every re-mirror (starred/Repo,
Repo-owner, Repo-owner-1, ...) because the existence check only asked
"does a repo with this name exist?" and never "is the existing repo a
mirror of THIS same source?". The repo's own prior mirror counted as a
collision, so generateUniqueRepoName bumped to the next suffix each run,
repointing mirroredLocation at the newest copy. Under a single re-call,
3 concurrent/retried jobs each computed a DIFFERENT suffixed name, so the
location-based in-flight guard never matched and the race produced extra
copies.

Fix (source-identity aware):
- New shared helper src/lib/utils/mirror-source-match.ts:
  - normalizeCloneUrl / cloneUrlsMatch: credential-, .git-, slash- and
    host-case-insensitive clone URL comparison.
  - isMirrorOfSource: a Gitea repo is "ours" only if it is a mirror AND
    its original_url matches this repo's source.
  - findExistingMirror: resolves an existing same-source mirror via the
    recorded mirroredLocation first (survives strategy changes — #309),
    then the base candidate name.
  - classifyCandidateName: pure available/reusable/taken decision.
- gitea-enhanced: export GiteaRepoInfo and add original_url (Gitea's
  recorded migration source) for source matching.
- Both create paths (mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg):
  run findExistingMirror BEFORE name generation; on a hit, reuse that
  location and route into the existing "already mirrored" handling rather
  than calling generateUniqueRepoName. Names now converge under
  concurrency so the in-flight guard becomes effective.
- generateUniqueRepoName is now source-aware: an occupied name held by a
  mirror of the SAME source is reused (no suffix); suffixing only happens
  on a genuine different-source collision, preserving #95/#236 behavior.
  The per-user DB claim check is retained so two users mirroring the same
  source into a shared org stay separated.
- Phantom-fork guard (#309): the existingRepoInfo.mirror branches now
  verify same-source before marking "mirrored"; on mismatch they fall
  through to unique-name generation and create a separate mirror.
- Scheduler: a `failed` repo whose mirroredLocation still resolves to a
  live same-source mirror is routed to syncGiteaRepo instead of re-create,
  breaking the failed-metadata re-create loop cheaply.
- Remove dead src/lib/starred-repos-handler.ts (zero importers across all
  git history); its correct base-name/.mirror reuse logic now lives in the
  shared helper.

Tests: src/lib/utils/mirror-source-match.test.ts (30 cases) covers URL
normalization, reuse at base name, reuse via mirroredLocation across a
strategy change, genuine different-source collision (suffix), phantom
fork, stale mirroredLocation fallback, per-user DB-claim separation, and
the suffix-vs-reuse classification. Full suite: 319 pass, 0 fail.
2026-06-13 08:00:41 +05:30
Arunavo Ray 699a5771f5 chore: bump version to 3.17.1 2026-06-05 18:42:11 +05:30
ARUNAVO RAY e862714d6a fix(db): self-heal sso_providers duplicate-column crash on upgrade (#312) (#313)
Migration 0013 runs as a single transaction that rebuilds `organizations`
and then `ALTER TABLE sso_providers ADD saml_config` / `ADD domain_verified`.
On instances where those columns were already created outside Drizzle (declared
in schema.ts and added via db:push / an SSO-register round-trip on an
intermediate build), the ADD throws "duplicate column name: saml_config". That
rolls back the entire 0013 transaction, so 0013 is never recorded in
`__drizzle_migrations` and is retried — failing identically — on every boot,
crash-looping the server.

Add a pre-migrate repair (mirroring the existing repairFailedMigrations() for
the 0009 case): when 0013 is unrecorded but the columns already exist, preserve
any real SAML provider config, drop the stranded columns so the canonical 0013
runs in full (organizations rebuild included), then restore the preserved
values once the columns are re-added. No-op on fresh installs, clean upgrades,
and already-migrated databases.

This lets affected instances recover automatically on the next boot after
upgrading — no manual SQLite surgery required.

- src/lib/db/migration-repairs.ts: repairDuplicateSsoColumns + restoreSsoDataAfter0013
- src/lib/db/index.ts: wire both around migrate()
- scripts/validate-migrations.ts: cover the broken-upgrade + data-preservation path
2026-06-05 18:41:46 +05:30
37 changed files with 4961 additions and 1943 deletions
+318 -316
View File
File diff suppressed because it is too large Load Diff
+3 -1
View File
@@ -107,6 +107,7 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
| `MIRROR_STARRED_LISTS` | Optional comma-separated GitHub Star List names to mirror (only used when `MIRROR_STARRED=true`) | - | Comma-separated list names (empty = all starred repos) |
| `STARRED_REPOS_ORG` | Organization name for starred repos | `starred` | Any string |
| `STARRED_REPOS_MODE` | How starred repos are mirrored | `dedicated-org` | `dedicated-org`, `preserve-owner` |
| `STARRED_DUPLICATE_STRATEGY` | Name collision strategy when two starred repos share a name from different owners (`suffix` = `repo-owner`, `prefix` = `owner-repo`) | `suffix` | `suffix`, `prefix`, `owner-org` |
### Organization Settings
@@ -114,7 +115,8 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
|----------|-------------|---------|---------|
| `MIRROR_ORGANIZATIONS` | Mirror organization repositories | `false` | `true`, `false` |
| `PRESERVE_ORG_STRUCTURE` | Preserve GitHub organization structure in Gitea | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal) | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal); sets `skipPersonalRepos: true` in GitHub config | `false` | `true`, `false` |
| `INCLUDE_ORGANIZATIONS` | Opt-in allowlist: only mirror repos from these organizations (empty = all orgs you belong to). Sets `includeOrganizations` in GitHub config | - | Comma-separated org names |
| `MIRROR_STRATEGY` | Repository organization strategy | `preserve` | `preserve`, `single-org`, `flat-user`, `mixed` |
### Advanced Settings
+43 -43
View File
@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.17.0",
"version": "3.20.4",
"engines": {
"bun": ">=1.2.9"
},
@@ -54,39 +54,39 @@
"picomatch": "^4.0.4"
},
"dependencies": {
"@astrojs/check": "^0.9.7",
"@astrojs/check": "^0.9.9",
"@astrojs/mdx": "5.0.0",
"@astrojs/node": "10.0.1",
"@astrojs/react": "^5.0.0",
"@astrojs/node": "10.1.4",
"@astrojs/react": "^5.0.7",
"@better-auth/oauth-provider": "1.6.11",
"@better-auth/sso": "1.6.11",
"@octokit/plugin-throttling": "^11.0.3",
"@octokit/rest": "^22.0.1",
"@radix-ui/react-accordion": "^1.2.12",
"@radix-ui/react-avatar": "^1.1.11",
"@radix-ui/react-checkbox": "^1.3.3",
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-dialog": "^1.1.15",
"@radix-ui/react-dropdown-menu": "^2.1.16",
"@radix-ui/react-hover-card": "^1.1.15",
"@radix-ui/react-label": "^2.1.8",
"@radix-ui/react-popover": "^1.1.15",
"@radix-ui/react-progress": "^1.1.8",
"@radix-ui/react-radio-group": "^1.3.8",
"@radix-ui/react-scroll-area": "^1.2.10",
"@radix-ui/react-select": "^2.2.6",
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
"@radix-ui/react-switch": "^1.2.6",
"@radix-ui/react-tabs": "^1.1.13",
"@radix-ui/react-tooltip": "^1.2.8",
"@tailwindcss/vite": "^4.2.1",
"@radix-ui/react-accordion": "^1.2.13",
"@radix-ui/react-avatar": "^1.1.12",
"@radix-ui/react-checkbox": "^1.3.4",
"@radix-ui/react-collapsible": "^1.1.13",
"@radix-ui/react-dialog": "^1.1.16",
"@radix-ui/react-dropdown-menu": "^2.1.17",
"@radix-ui/react-hover-card": "^1.1.16",
"@radix-ui/react-label": "^2.1.9",
"@radix-ui/react-popover": "^1.1.16",
"@radix-ui/react-progress": "^1.1.9",
"@radix-ui/react-radio-group": "^1.4.0",
"@radix-ui/react-scroll-area": "^1.2.11",
"@radix-ui/react-select": "^2.3.0",
"@radix-ui/react-separator": "^1.1.9",
"@radix-ui/react-slot": "^1.2.5",
"@radix-ui/react-switch": "^1.3.0",
"@radix-ui/react-tabs": "^1.1.14",
"@radix-ui/react-tooltip": "^1.2.9",
"@tailwindcss/vite": "^4.3.1",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.19",
"@tanstack/react-virtual": "^3.14.2",
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.2.14",
"@types/react": "^19.2.17",
"@types/react-dom": "^19.2.3",
"astro": "^6.0.4",
"astro": "^6.4.6",
"bcryptjs": "^3.0.3",
"better-auth": "1.6.11",
"buffer": "^6.0.3",
@@ -94,39 +94,39 @@
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"dotenv": "^17.3.1",
"dotenv": "^17.4.2",
"drizzle-orm": "^0.45.2",
"fuse.js": "^7.1.0",
"fuse.js": "^7.4.2",
"jsonwebtoken": "^9.0.3",
"lucide-react": "^0.577.0",
"nanoid": "^5.1.6",
"nanoid": "^5.1.11",
"next-themes": "^0.4.6",
"react": "^19.2.4",
"react-dom": "^19.2.4",
"react-icons": "^5.5.0",
"react": "^19.2.7",
"react-dom": "^19.2.7",
"react-icons": "^5.6.0",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.2.1",
"tailwind-merge": "^3.6.0",
"tailwindcss": "^4.3.1",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.3",
"uuid": "^13.0.0",
"uuid": "^13.0.2",
"vaul": "^1.1.2",
"zod": "^4.3.6"
"zod": "^4.4.3"
},
"devDependencies": {
"@playwright/test": "^1.58.2",
"@playwright/test": "^1.60.0",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^16.3.2",
"@types/bcryptjs": "^3.0.0",
"@types/bun": "^1.3.10",
"@types/bun": "^1.3.14",
"@types/jsonwebtoken": "^9.0.10",
"@types/node": "^25.5.0",
"@types/node": "^25.9.3",
"@types/uuid": "^11.0.0",
"@vitejs/plugin-react": "^6.0.1",
"drizzle-kit": "^0.31.9",
"@vitejs/plugin-react": "^6.0.2",
"drizzle-kit": "^0.31.10",
"jsdom": "^28.1.0",
"tsx": "^4.21.0",
"vitest": "^4.1.0"
"tsx": "^4.22.4",
"vitest": "^4.1.8"
},
"packageManager": "bun@1.3.10"
}
+102 -1
View File
@@ -3,6 +3,10 @@
import { Database } from "bun:sqlite";
import { readFileSync } from "fs";
import path from "path";
import {
repairDuplicateSsoColumns,
restoreSsoDataAfter0013,
} from "../src/lib/db/migration-repairs";
type JournalEntry = {
idx: number;
@@ -290,6 +294,100 @@ function verify0013Migration(db: any) {
assert(normName.dflt_value === null, `Expected normalized_name to have no default, got ${normName.dflt_value}`);
}
const MIGRATION_0012_TIMESTAMP = 1774062000000;
const MIGRATION_0013_TIMESTAMP = 1780377747526;
/**
* Reproduce the issue #312 crash state — sso_providers already carries
* saml_config / domain_verified before migration 0013 runs (stranded on an
* intermediate build), with __drizzle_migrations recorded only through 0012 —
* and verify repairDuplicateSsoColumns()/restoreSsoDataAfter0013() let the
* canonical 0013 run while preserving real SAML provider data.
*/
function validateBroken0013Repair() {
const migration0013 = migrations.find((m) => m.entry.tag === "0013_slim_galactus");
if (!migration0013) return; // 0013 not present (shouldn't happen) — nothing to test.
const db = new Database(":memory:");
try {
runMigrations(db, migrations.slice(0, 13)); // 0000-0012
// A real upgraded instance has a __drizzle_migrations table recorded
// through 0012 but not 0013.
db.run(
"CREATE TABLE IF NOT EXISTS `__drizzle_migrations` (id INTEGER PRIMARY KEY AUTOINCREMENT, hash text NOT NULL, created_at numeric)",
);
db.run("INSERT INTO `__drizzle_migrations` (hash, created_at) VALUES ('through-0012', ?)", [
MIGRATION_0012_TIMESTAMP,
]);
// Stranded columns from the intermediate build.
db.run("ALTER TABLE sso_providers ADD saml_config text");
db.run("ALTER TABLE sso_providers ADD domain_verified integer DEFAULT true NOT NULL");
db.run("INSERT INTO users (id, email, username, name) VALUES ('u1', 'u1@example.com', 'u1', 'User One')");
const samlJson = '{"entryPoint":"https://idp.example.com/sso","cert":"ABC123"}';
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('oidc1', 'https://idp', 'a.com', '{}', 'u1', 'p-oidc', NULL, 1)",
);
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('saml1', 'https://idp', 'b.com', '{}', 'u1', 'p-saml', ?, 1)",
[samlJson],
);
db.run(
"INSERT INTO sso_providers (id, issuer, domain, oidc_config, user_id, provider_id, saml_config, domain_verified) VALUES ('unv1', 'https://idp', 'c.com', '{}', 'u1', 'p-unv', NULL, 0)",
);
const preserved = repairDuplicateSsoColumns(db);
const colsAfterRepair = (db.query("PRAGMA table_info(sso_providers)").all() as TableInfoRow[]).map(
(c) => c.name,
);
assert(!colsAfterRepair.includes("saml_config"), "Expected repair to drop stranded saml_config column");
assert(
!colsAfterRepair.includes("domain_verified"),
"Expected repair to drop stranded domain_verified column",
);
const preservedIds = preserved.map((r) => r.id).sort();
assert(
preservedIds.length === 2 && preservedIds[0] === "saml1" && preservedIds[1] === "unv1",
`Expected SAML + unverified rows to be preserved, got ${JSON.stringify(preservedIds)}`,
);
// The canonical 0013 must now run without a duplicate-column error.
runMigration(db, migration0013);
restoreSsoDataAfter0013(db, preserved);
const rows = db
.query("SELECT id, saml_config, domain_verified FROM sso_providers ORDER BY id")
.all() as Array<{ id: string; saml_config: string | null; domain_verified: number }>;
const byId = Object.fromEntries(rows.map((r) => [r.id, r]));
assert(byId.oidc1.saml_config === null, "Expected OIDC provider saml_config to remain NULL");
assert(byId.oidc1.domain_verified === 1, "Expected OIDC provider domain_verified default 1");
assert(byId.saml1.saml_config === samlJson, "Expected SAML provider config to be preserved");
assert(byId.saml1.domain_verified === 1, "Expected SAML provider domain_verified preserved as 1");
assert(byId.unv1.saml_config === null, "Expected unverified provider saml_config NULL");
assert(byId.unv1.domain_verified === 0, "Expected explicit domain_verified=0 to be preserved");
// Idempotency: 0013 is now applied, so a re-run of the repair is a no-op.
db.run("INSERT INTO `__drizzle_migrations` (hash, created_at) VALUES ('through-0013', ?)", [
MIGRATION_0013_TIMESTAMP,
]);
const secondPass = repairDuplicateSsoColumns(db);
assert(secondPass.length === 0, "Expected repair to no-op once migration 0013 is recorded");
const colsAfterSecondPass = (
db.query("PRAGMA table_info(sso_providers)").all() as TableInfoRow[]
).map((c) => c.name);
assert(
colsAfterSecondPass.includes("saml_config") && colsAfterSecondPass.includes("domain_verified"),
"Expected columns to remain intact on the no-op second pass",
);
} finally {
db.close();
}
}
const latestUpgradeFixtures: Record<string, UpgradeFixture> = {
"0009_nervous_tyger_tiger": {
seed: seedPre0009Database,
@@ -361,8 +459,11 @@ function validateMigrations() {
upgradeDb.close();
}
// Exercise the runtime repair for the issue #312 duplicate-column crash.
validateBroken0013Repair();
console.log(
`Validated ${migrations.length} migrations from scratch and upgrade path for ${latestMigration.entry.tag}.`,
`Validated ${migrations.length} migrations from scratch and upgrade path for ${latestMigration.entry.tag}, plus the #312 SSO-column repair.`,
);
}
+97 -7
View File
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useState, useRef } from 'react';
import { Button } from '@/components/ui/button';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter } from 'lucide-react';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter, StopCircle } from 'lucide-react';
import {
DropdownMenu,
DropdownMenuContent,
@@ -85,6 +85,8 @@ export function ActivityLog() {
const [activities, setActivities] = useState<MirrorJobWithKey[]>([]);
const [isInitialLoading, setIsInitialLoading] = useState(false);
const [showCleanupDialog, setShowCleanupDialog] = useState(false);
const [showCancelPendingDialog, setShowCancelPendingDialog] = useState(false);
const [isCancelPendingLoading, setIsCancelPendingLoading] = useState(false);
// Ref to track if component is mounted to prevent state updates after unmount
const isMountedRef = useRef(true);
@@ -354,6 +356,40 @@ export function ActivityLog() {
setShowCleanupDialog(false);
};
const confirmCancelPending = async () => {
if (!user?.id) return;
try {
setIsCancelPendingLoading(true);
setShowCancelPendingDialog(false);
const response = await fetch(withBase('/api/job/cancel-pending'), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
});
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: 'Unknown error occurred' }));
throw new Error(errorData.error || `HTTP ${response.status}: ${response.statusText}`);
}
const res = await response.json();
if (res.success) {
toast.success(res.message);
// Refresh to show the new activity log entry
await fetchActivities(false);
} else {
showErrorToast(res.error || 'Failed to cancel pending mirrors.', toast);
}
} catch (error) {
console.error('Error cancelling pending mirrors:', error);
showErrorToast(error, toast);
} finally {
setIsCancelPendingLoading(false);
}
};
// Check if any filters are active
const hasActiveFilters = !!(filter.status || filter.type || filter.name);
const activeFilterCount = [filter.status, filter.type, filter.name].filter(Boolean).length;
@@ -552,11 +588,22 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10 shrink-0"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10 shrink-0"
>
<Trash2 className="h-4 w-4" />
@@ -683,12 +730,24 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
{/* cleanup all activities */}
{/* stop pending mirrors */}
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
{/* clear activity history */}
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10"
>
<Trash2 className="h-4 w-4" />
@@ -709,9 +768,12 @@ export function ActivityLog() {
<Dialog open={showCleanupDialog} onOpenChange={setShowCleanupDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete All Activities</DialogTitle>
<DialogTitle>Clear Activity History</DialogTitle>
<DialogDescription>
Are you sure you want to delete ALL activities? This action cannot be undone and will remove all mirror jobs and events from the database.
This clears the activity <strong>history log</strong> (mirror job records and events) it does not stop
any pending or in-progress mirrors. Repositories keep their current status and the scheduler
will continue to pick up pending work. To stop pending mirrors, use the
&ldquo;Stop Pending Mirrors&rdquo; button instead.
</DialogDescription>
</DialogHeader>
<DialogFooter>
@@ -723,7 +785,35 @@ export function ActivityLog() {
onClick={confirmCleanup}
disabled={isInitialLoading}
>
{isInitialLoading ? 'Deleting...' : 'Delete All Activities'}
{isInitialLoading ? 'Clearing...' : 'Clear History'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* cancel pending mirrors confirmation dialog */}
<Dialog open={showCancelPendingDialog} onOpenChange={setShowCancelPendingDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Stop Pending Mirrors</DialogTitle>
<DialogDescription>
This sets all repositories with status <strong>Imported</strong> or <strong>Failed</strong> to{' '}
<strong>Ignored</strong>, preventing the scheduler from mirroring them automatically.
Repositories that are currently mirroring are not affected.{' '}
You can re-enable individual repositories from the Repositories page.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button variant="outline" onClick={() => setShowCancelPendingDialog(false)}>
Cancel
</Button>
<Button
variant="default"
className="bg-amber-600 hover:bg-amber-700"
onClick={confirmCancelPending}
disabled={isCancelPendingLoading}
>
{isCancelPendingLoading ? 'Stopping...' : 'Stop Pending Mirrors'}
</Button>
</DialogFooter>
</DialogContent>
@@ -77,6 +77,7 @@ export function GitHubMirrorSettings({
const [starListsOpen, setStarListsOpen] = React.useState(false);
const [starListSearch, setStarListSearch] = React.useState("");
const [customStarListName, setCustomStarListName] = React.useState("");
const [customOrgName, setCustomOrgName] = React.useState("");
const [availableStarLists, setAvailableStarLists] = React.useState<string[]>([]);
const [loadingStarLists, setLoadingStarLists] = React.useState(false);
const [loadedStarLists, setLoadedStarLists] = React.useState(false);
@@ -138,6 +139,38 @@ export function GitHubMirrorSettings({
});
}, [githubConfig, normalizeStarListNames, onGitHubConfigChange]);
const includedOrgs = React.useMemo(
() => githubConfig.includeOrganizations ?? [],
[githubConfig.includeOrganizations],
);
const addIncludedOrg = React.useCallback(() => {
const trimmed = customOrgName.trim();
if (!trimmed) return;
const exists = includedOrgs.some(
(org) => org.toLowerCase() === trimmed.toLowerCase(),
);
if (!exists) {
onGitHubConfigChange({
...githubConfig,
includeOrganizations: [...includedOrgs, trimmed],
});
}
setCustomOrgName("");
}, [customOrgName, includedOrgs, githubConfig, onGitHubConfigChange]);
const removeIncludedOrg = React.useCallback(
(name: string) => {
onGitHubConfigChange({
...githubConfig,
includeOrganizations: includedOrgs.filter(
(org) => org.toLowerCase() !== name.toLowerCase(),
),
});
},
[includedOrgs, githubConfig, onGitHubConfigChange],
);
const loadStarLists = React.useCallback(async () => {
if (
loadingStarLists ||
@@ -927,6 +960,84 @@ export function GitHubMirrorSettings({
</p>
</div>
</div>
<div className="flex items-start space-x-3">
<Checkbox
id="skip-personal-repos"
checked={advancedOptions.skipPersonalRepos ?? false}
onCheckedChange={(checked) => handleAdvancedChange('skipPersonalRepos', !!checked)}
/>
<div className="space-y-0.5 flex-1">
<Label
htmlFor="skip-personal-repos"
className="text-sm font-normal cursor-pointer flex items-center gap-2"
>
<Users className="h-3.5 w-3.5" />
Skip personal repositories (only mirror organization repos)
</Label>
<p className="text-xs text-muted-foreground">
Exclude repositories owned by your personal GitHub account; only mirror repos belonging to organizations
</p>
</div>
</div>
<div className="flex items-start space-x-3">
<Users className="h-4 w-4 mt-0.5 text-muted-foreground" />
<div className="space-y-2 flex-1">
<div className="space-y-0.5">
<Label className="text-sm font-normal flex items-center gap-2">
Limit to specific organizations
</Label>
<p className="text-xs text-muted-foreground">
Leave empty to mirror repos from every organization you belong to.
Add one or more organizations to mirror only their repositories.
</p>
</div>
{includedOrgs.length > 0 && (
<div className="flex flex-wrap gap-1.5">
{includedOrgs.map((org) => (
<Badge key={org} variant="secondary" className="gap-1">
<span>{org}</span>
<button
type="button"
onClick={() => removeIncludedOrg(org)}
className="rounded-sm hover:text-foreground/80"
aria-label={`Remove ${org} organization`}
>
<X className="h-3 w-3" />
</button>
</Badge>
))}
</div>
)}
<div className="flex items-center gap-2">
<Input
value={customOrgName}
onChange={(event) => setCustomOrgName(event.target.value)}
onKeyDown={(event) => {
if (event.key === "Enter") {
event.preventDefault();
addIncludedOrg();
}
}}
placeholder="Add organization name"
className="h-8 text-xs"
/>
<Button
type="button"
variant="outline"
size="sm"
className="h-8"
onClick={addIncludedOrg}
disabled={!customOrgName.trim()}
>
Add
</Button>
</div>
</div>
</div>
</div>
</div>
</div>
+189 -106
View File
@@ -98,6 +98,8 @@ export default function Repository() {
const [repoToDelete, setRepoToDelete] = useState<Repository | null>(null);
const [isDeleteRepoDialogOpen, setIsDeleteRepoDialogOpen] = useState(false);
const [isDeletingRepo, setIsDeletingRepo] = useState(false);
const [isBulkDeleteDialogOpen, setIsBulkDeleteDialogOpen] = useState(false);
const [isDeletingBulk, setIsDeletingBulk] = useState(false);
// Create a stable callback using useCallback
const handleNewMessage = useCallback((data: MirrorJob) => {
@@ -919,11 +921,9 @@ export default function Repository() {
setIsDeletingRepo(true);
try {
const response = await apiRequest<{ success: boolean; error?: string }>(
`/repositories/${repoToDelete.id}`,
{
method: "DELETE",
}
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [repoToDelete.id] }) }
);
if (response.success) {
@@ -941,6 +941,30 @@ export default function Repository() {
}
};
const handleBulkDelete = async () => {
if (!user || !user.id) return;
setIsDeletingBulk(true);
try {
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [...selectedRepoIds] }) }
);
if (response.success) {
const count = response.deleted ?? selectedRepoIds.size;
toast.success(`Removed ${count} ${count === 1 ? "repository" : "repositories"} from Gitea Mirror.`);
setSelectedRepoIds(new Set());
await fetchRepositories(false);
} else {
showErrorToast(response.error || "Failed to delete repositories", toast);
}
} catch (error) {
showErrorToast(error, toast);
} finally {
setIsDeletingBulk(false);
setIsBulkDeleteDialogOpen(false);
}
};
// Determine what actions are available for selected repositories
const getAvailableActions = () => {
if (selectedRepoIds.size === 0) return [];
@@ -977,7 +1001,9 @@ export default function Repository() {
if (selectedRepos.some(repo => repo.status === "ignored")) {
actions.push('include');
}
actions.push('delete');
return actions;
};
@@ -994,6 +1020,7 @@ export default function Repository() {
retry: selectedRepos.filter(repo => repo.status === "failed").length,
ignore: selectedRepos.filter(repo => repo.status !== "ignored").length,
include: selectedRepos.filter(repo => repo.status === "ignored").length,
delete: selectedRepos.length,
};
};
@@ -1312,111 +1339,122 @@ export default function Repository() {
</Button>
</div>
{/* Bulk actions on desktop - integrated into the same line */}
{/* Mirror All action */}
<div className="flex items-center gap-2 border-l pl-4">
{selectedRepoIds.size === 0 ? (
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
) : (
<>
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
</>
)}
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
</div>
</div>
</div>
{/* Desktop: Bulk actions row - shown when repos are selected */}
{selectedRepoIds.size > 0 && (
<div className="hidden sm:flex items-center gap-2 flex-wrap">
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
<Button
variant="destructive"
size="default"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
)}
{/* Action buttons for mobile - only show when items are selected */}
{selectedRepoIds.size > 0 && (
<div className="flex items-center gap-2 flex-wrap sm:hidden">
@@ -1506,6 +1544,16 @@ export default function Repository() {
Include
</Button>
)}
<Button
variant="destructive"
size="sm"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
</div>
)}
@@ -1586,6 +1634,41 @@ export default function Repository() {
</DialogContent>
</Dialog>
<Dialog
open={isBulkDeleteDialogOpen}
onOpenChange={(open) => {
if (!open && !isDeletingBulk) setIsBulkDeleteDialogOpen(false);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Remove {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"} from Gitea Mirror?</DialogTitle>
<DialogDescription>
These repositories will be deleted from Gitea Mirror only. Any mirrors on Gitea will remain untouched; remove them manually in Gitea if needed.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => setIsBulkDeleteDialogOpen(false)}
disabled={isDeletingBulk}
>
Cancel
</Button>
<Button variant="destructive" onClick={handleBulkDelete} disabled={isDeletingBulk}>
{isDeletingBulk ? (
<LoaderCircle className="h-4 w-4 animate-spin" />
) : (
<span className="flex items-center gap-2">
<Trash2 className="h-4 w-4" />
Delete {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"}
</span>
)}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog
open={isDeleteRepoDialogOpen}
onOpenChange={(open) => {
+57 -12
View File
@@ -99,6 +99,7 @@ export default function RepositoryTable({
onDismissSync,
}: RepositoryTableProps) {
const tableParentRef = useRef<HTMLDivElement>(null);
const lastSelectedIndexRef = useRef<number | null>(null);
const { giteaConfig } = useGiteaConfig();
const handleUpdateDestination = async (repoId: string, newDestination: string | null) => {
@@ -235,6 +236,7 @@ export default function RepositoryTable({
// Selection handlers
const handleSelectAll = (checked: boolean) => {
lastSelectedIndexRef.current = null;
if (checked) {
const allIds = new Set(
visibleRepositories
@@ -247,7 +249,7 @@ export default function RepositoryTable({
}
};
const handleSelectRepo = (repoId: string, checked: boolean) => {
const handleSelectRepo = (repoId: string, checked: boolean, index?: number) => {
const newSelection = new Set(selectedRepoIds);
if (checked) {
newSelection.add(repoId);
@@ -255,6 +257,32 @@ export default function RepositoryTable({
newSelection.delete(repoId);
}
onSelectionChange(newSelection);
if (index !== undefined) {
lastSelectedIndexRef.current = index;
}
};
const handleShiftRangeSelect = (currentIndex: number) => {
const lastIndex = lastSelectedIndexRef.current;
if (lastIndex === null) {
const repo = visibleRepositories[currentIndex];
if (repo?.id) {
const newSelection = new Set(selectedRepoIds);
newSelection.add(repo.id);
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
}
return;
}
const start = Math.min(lastIndex, currentIndex);
const end = Math.max(lastIndex, currentIndex);
const newSelection = new Set(selectedRepoIds);
for (let i = start; i <= end; i++) {
const id = visibleRepositories[i]?.id;
if (id) newSelection.add(id);
}
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
};
const isAllSelected =
@@ -263,7 +291,7 @@ export default function RepositoryTable({
const isPartiallySelected = selectedRepoIds.size > 0 && !isAllSelected;
// Mobile card layout for repository
const RepositoryCard = ({ repo }: { repo: Repository }) => {
const RepositoryCard = ({ repo, index }: { repo: Repository; index: number }) => {
const isLoading = repo.id ? loadingRepoIds.has(repo.id) : false;
const isSelected = repo.id ? selectedRepoIds.has(repo.id) : false;
const giteaUrl = getGiteaRepoUrl(repo);
@@ -274,12 +302,21 @@ export default function RepositoryTable({
<div className="flex flex-col gap-3">
{/* Header with checkbox and repo name */}
<div className="flex items-start gap-3">
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
<div
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(index);
}
}}
>
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean, index)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
</div>
<div className="flex-1 min-w-0">
<h3 className="font-medium text-base truncate">{repo.name}</h3>
<div className="flex items-center gap-2 mt-1 flex-wrap">
@@ -635,8 +672,8 @@ export default function RepositoryTable({
</div>
{/* Repository cards */}
{visibleRepositories.map((repo) => (
<RepositoryCard key={repo.id} repo={repo} />
{visibleRepositories.map((repo, index) => (
<RepositoryCard key={repo.id} repo={repo} index={index} />
))}
</div>
@@ -701,10 +738,18 @@ export default function RepositoryTable({
className="h-[65px] flex items-center justify-between bg-transparent border-b hover:bg-muted/50"
>
{/* Checkbox */}
<div className="h-full p-3 flex items-center justify-center flex-[0.3]">
<div
className="h-full p-3 flex items-center justify-center flex-[0.3]"
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(virtualRow.index);
}
}}
>
<Checkbox
checked={repo.id ? selectedRepoIds.has(repo.id) : false}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked)}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked, virtualRow.index)}
aria-label={`Select ${repo.name}`}
/>
</div>
+9
View File
@@ -3,6 +3,7 @@ import { drizzle } from "drizzle-orm/bun-sqlite";
import fs from "fs";
import path from "path";
import { migrate } from "drizzle-orm/bun-sqlite/migrator";
import { repairDuplicateSsoColumns, restoreSsoDataAfter0013 } from "./migration-repairs";
// Skip database initialization in test environment
let db: ReturnType<typeof drizzle>;
@@ -95,9 +96,17 @@ if (process.env.NODE_ENV !== "test") {
// Fix any migrations that were recorded but actually failed (e.g. v3.13.0 bug)
repairFailedMigrations();
// Fix the v3.17.0 duplicate-column crash: reconcile stranded sso_providers
// columns so migration 0013 can run (see #312). Returns data to re-apply
// once 0013 has re-added the columns.
const preservedSsoData = repairDuplicateSsoColumns(sqlite);
// Run migrations using Drizzle migrate function
migrate(db, { migrationsFolder: "./drizzle" });
// Re-apply any SSO provider data preserved by the repair above.
restoreSsoDataAfter0013(sqlite, preservedSsoData);
console.log("✅ Database migrations completed successfully");
} catch (error) {
console.error("❌ Error running migrations:", error);
+117
View File
@@ -0,0 +1,117 @@
import type { Database } from "bun:sqlite";
/**
* Pre-migration repairs that reconcile a database into the exact shape Drizzle's
* migrator expects, so a previously-failed migration can complete on the next
* boot. These run BEFORE `migrate()` and are deliberately defensive: any failure
* is logged and swallowed so they never make a recoverable database worse.
*/
/** Migration 0013 journal timestamp (from drizzle/meta/_journal.json, idx 13). */
const MIGRATION_0013_TIMESTAMP = 1780377747526;
export type PreservedSsoRow = {
id: string;
saml_config?: string | null;
domain_verified?: number;
};
/**
* Repair the v3.17.0 (PR #307) "duplicate column name: saml_config" crash loop
* reported in issue #312.
*
* Some instances ended up with `sso_providers.saml_config` / `domain_verified`
* already present BEFORE migration 0013 ran the columns were declared in
* schema.ts and entered the DB via `db:push` or an SSO-register round-trip on an
* intermediate build, while `__drizzle_migrations` never recorded a 0013 row.
*
* Migration 0013 runs as a single transaction (organizations rebuild + the two
* `ALTER TABLE sso_providers ADD ...`). The ADD hits the pre-existing column,
* throws "duplicate column", and rolls back the ENTIRE transaction so 0013 is
* never recorded and is retried, failing identically, on every boot.
*
* This is the mirror image of the 0009 repair in index.ts (record present,
* column missing): here the column is present but the record is missing. We
* reconcile `sso_providers` back to its true pre-0013 shape so the canonical
* 0013 can run in full (the organizations rebuild MUST NOT be skipped),
* preserving any real SAML provider config across the drop/re-add.
*
* Returns the rows whose values must be re-applied by {@link restoreSsoDataAfter0013}
* once 0013 has re-added the columns. Returns an empty array when there is
* nothing to do (fresh install, clean upgrade, or genuine pre-0013 shape).
*/
export function repairDuplicateSsoColumns(sqlite: Database): PreservedSsoRow[] {
try {
const migrationsTableExists = sqlite
.query("SELECT name FROM sqlite_master WHERE type='table' AND name='__drizzle_migrations'")
.get();
// Fresh install — no migrations recorded yet, vanilla migrate() handles it.
if (!migrationsTableExists) return [];
// 0013 already recorded (clean upgrade / already healed) — nothing to do.
const alreadyApplied = sqlite
.query("SELECT 1 FROM __drizzle_migrations WHERE created_at >= ? LIMIT 1")
.get(MIGRATION_0013_TIMESTAMP);
if (alreadyApplied) return [];
const ssoExists = sqlite
.query("SELECT name FROM sqlite_master WHERE type='table' AND name='sso_providers'")
.get();
if (!ssoExists) return [];
const cols = sqlite.query("PRAGMA table_info(sso_providers)").all() as { name: string }[];
const hasSaml = cols.some((c) => c.name === "saml_config");
const hasDomainVerified = cols.some((c) => c.name === "domain_verified");
// Genuine pre-0013 shape — let migration 0013 add the columns as-is.
if (!hasSaml && !hasDomainVerified) return [];
console.log(
"🔧 Detected stranded SSO columns (migration 0013 not recorded). Reconciling sso_providers so 0013 can run...",
);
// Preserve any real data before dropping. SAML providers store JSON config
// in saml_config; domain_verified may have been explicitly set to false.
const selectCols = ["id"];
if (hasSaml) selectCols.push("saml_config");
if (hasDomainVerified) selectCols.push("domain_verified");
const preserved = sqlite
.query(`SELECT ${selectCols.join(", ")} FROM sso_providers`)
.all() as PreservedSsoRow[];
// SQLite >= 3.35 (bun:sqlite ships much newer) supports DROP COLUMN.
if (hasSaml) sqlite.run("ALTER TABLE sso_providers DROP COLUMN saml_config");
if (hasDomainVerified) sqlite.run("ALTER TABLE sso_providers DROP COLUMN domain_verified");
// Only rows whose values differ from the 0013 defaults (saml_config NULL,
// domain_verified true/1) need restoring after the columns are re-added.
return preserved.filter(
(r) => (hasSaml && r.saml_config != null) || (hasDomainVerified && r.domain_verified === 0),
);
} catch (error) {
console.warn("⚠️ SSO column repair check failed (non-fatal):", error);
return [];
}
}
/**
* Re-apply the SSO provider values preserved by {@link repairDuplicateSsoColumns}
* once migration 0013 has re-added saml_config / domain_verified with their
* defaults (saml_config NULL, domain_verified = 1). No-op when nothing was
* preserved (the common OIDC-only case).
*/
export function restoreSsoDataAfter0013(sqlite: Database, preserved: PreservedSsoRow[]): void {
if (preserved.length === 0) return;
try {
const stmt = sqlite.prepare(
"UPDATE sso_providers SET saml_config = ?, domain_verified = ? WHERE id = ?",
);
for (const r of preserved) {
stmt.run(r.saml_config ?? null, r.domain_verified ?? 1, r.id);
}
console.log(`✅ Restored ${preserved.length} preserved SSO provider value(s) after migration 0013.`);
} catch (error) {
console.warn("⚠️ Failed to restore preserved SSO data (non-fatal):", error);
}
}
+3
View File
@@ -34,6 +34,7 @@ export const githubConfigSchema = z.object({
autoMirrorStarred: z.boolean().default(false),
skipStarredIssues: z.boolean().optional(), // Deprecated: kept for backward compatibility, use starredCodeOnly instead
starredDuplicateStrategy: z.enum(["suffix", "prefix", "owner-org"]).default("suffix").optional(),
skipPersonalRepos: z.boolean().default(false),
});
export const backupStrategyEnum = z.enum([
@@ -156,7 +157,9 @@ export const configSchema = z.object({
isActive: z.boolean().default(true),
githubConfig: githubConfigSchema,
giteaConfig: giteaConfigSchema,
// Unused/reserved — stored for future glob support but not currently read
include: z.array(z.string()).default(["*"]),
// Unused/reserved — stored for future glob support but not currently read
exclude: z.array(z.string()).default([]),
scheduleConfig: scheduleConfigSchema,
cleanupConfig: cleanupConfigSchema,
+13 -1
View File
@@ -20,12 +20,14 @@ interface EnvConfig {
skipForks?: boolean;
includeArchived?: boolean;
mirrorOrganizations?: boolean;
includeOrganizations?: string[];
preserveOrgStructure?: boolean;
onlyMirrorOrgs?: boolean;
starredCodeOnly?: boolean;
autoMirrorStarred?: boolean;
starredReposOrg?: string;
starredReposMode?: 'dedicated-org' | 'preserve-owner';
starredDuplicateStrategy?: 'suffix' | 'prefix' | 'owner-org';
starredLists?: string[];
mirrorStrategy?: 'preserve' | 'single-org' | 'flat-user' | 'mixed';
};
@@ -101,6 +103,9 @@ function parseEnvConfig(): EnvConfig {
const protectedRepos = process.env.CLEANUP_PROTECTED_REPOS
? process.env.CLEANUP_PROTECTED_REPOS.split(',').map(r => r.trim()).filter(Boolean)
: undefined;
const includeOrganizations = process.env.INCLUDE_ORGANIZATIONS
? process.env.INCLUDE_ORGANIZATIONS.split(',').map((org) => org.trim()).filter(Boolean)
: undefined;
const starredLists = process.env.MIRROR_STARRED_LISTS
? process.env.MIRROR_STARRED_LISTS.split(',').map((list) => list.trim()).filter(Boolean)
: undefined;
@@ -121,12 +126,14 @@ function parseEnvConfig(): EnvConfig {
skipForks: process.env.SKIP_FORKS === 'true',
includeArchived: process.env.INCLUDE_ARCHIVED === 'true',
mirrorOrganizations: process.env.MIRROR_ORGANIZATIONS === 'true',
includeOrganizations,
preserveOrgStructure: process.env.PRESERVE_ORG_STRUCTURE === 'true',
onlyMirrorOrgs: process.env.ONLY_MIRROR_ORGS === 'true',
starredCodeOnly: process.env.SKIP_STARRED_ISSUES === 'true',
autoMirrorStarred: process.env.AUTO_MIRROR_STARRED === 'true',
starredReposOrg: process.env.STARRED_REPOS_ORG,
starredReposMode: process.env.STARRED_REPOS_MODE as 'dedicated-org' | 'preserve-owner',
starredDuplicateStrategy: process.env.STARRED_DUPLICATE_STRATEGY as 'suffix' | 'prefix' | 'owner-org',
starredLists,
mirrorStrategy: process.env.MIRROR_STRATEGY as 'preserve' | 'single-org' | 'flat-user' | 'mixed',
},
@@ -277,14 +284,19 @@ export async function initializeConfigFromEnv(): Promise<void> {
includePrivate: envConfig.github.privateRepositories ?? existingConfig?.[0]?.githubConfig?.includePrivate ?? false,
includePublic: envConfig.github.publicRepositories ?? existingConfig?.[0]?.githubConfig?.includePublic ?? true,
includeCollaboratorRepos: envConfig.github.includeCollaboratorRepos ?? existingConfig?.[0]?.githubConfig?.includeCollaboratorRepos ?? true,
includeOrganizations: envConfig.github.mirrorOrganizations ? [] : (existingConfig?.[0]?.githubConfig?.includeOrganizations ?? []),
// Opt-in org allowlist from INCLUDE_ORGANIZATIONS (comma-separated). Falls
// back to existing config so the UI-managed list isn't clobbered on restart.
includeOrganizations: envConfig.github.includeOrganizations ?? existingConfig?.[0]?.githubConfig?.includeOrganizations ?? [],
starredReposOrg: envConfig.github.starredReposOrg || existingConfig?.[0]?.githubConfig?.starredReposOrg || 'starred',
starredReposMode: envConfig.github.starredReposMode || existingConfig?.[0]?.githubConfig?.starredReposMode || 'dedicated-org',
starredDuplicateStrategy: envConfig.github.starredDuplicateStrategy || existingConfig?.[0]?.githubConfig?.starredDuplicateStrategy || 'suffix',
mirrorStrategy,
defaultOrg: envConfig.gitea.organization || existingConfig?.[0]?.githubConfig?.defaultOrg || 'github-mirrors',
starredCodeOnly: envConfig.github.starredCodeOnly ?? existingConfig?.[0]?.githubConfig?.starredCodeOnly ?? false,
autoMirrorStarred: envConfig.github.autoMirrorStarred ?? existingConfig?.[0]?.githubConfig?.autoMirrorStarred ?? false,
starredLists: envConfig.github.starredLists ?? existingConfig?.[0]?.githubConfig?.starredLists ?? [],
// ONLY_MIRROR_ORGS=true maps to skipPersonalRepos: true
skipPersonalRepos: envConfig.github.onlyMirrorOrgs ?? existingConfig?.[0]?.githubConfig?.skipPersonalRepos ?? false,
};
// Build Gitea config
+245
View File
@@ -0,0 +1,245 @@
/**
* Unit tests for archiveGiteaRepo's return value and sanitizeRepoNameAlphaDashDot
* regression coverage for #331's follow-up (repos falsely flagged as orphaned
* and archived, then unreachable by "Manual Sync" because the DB's
* mirroredLocation/name were never updated to the post-rename name).
*
* archiveGiteaRepo now reports the Gitea-side name it ended up with after a
* rename (mirror path) so callers (repository-cleanup-service.ts) can persist
* it, instead of leaving the DB pointing at a name that no longer exists.
*/
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
const mockHttpGet = mock(async (_url: string, _headers?: any) => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPatch = mock(async (_url: string, _body?: any, _headers?: any) => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPost = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpDelete = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPut = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
class MockHttpError extends Error {
constructor(
message: string,
public status: number,
public statusText: string,
public response?: string
) {
super(message);
this.name = "HttpError";
}
}
mock.module("@/lib/http-client", () => ({
httpGet: mockHttpGet,
httpPatch: mockHttpPatch,
httpPost: mockHttpPost,
httpDelete: mockHttpDelete,
httpPut: mockHttpPut,
HttpError: MockHttpError,
}));
import { archiveGiteaRepo, sanitizeRepoNameAlphaDashDot } from "./gitea";
describe("sanitizeRepoNameAlphaDashDot", () => {
test("replaces disallowed characters with a dash", () => {
expect(sanitizeRepoNameAlphaDashDot("my repo!")).toBe("my-repo");
});
test("collapses consecutive disallowed characters into a single dash", () => {
expect(sanitizeRepoNameAlphaDashDot("a___b")).toBe("a-b");
});
test("trims leading and trailing separators/dots", () => {
expect(sanitizeRepoNameAlphaDashDot("--.foo.--")).toBe("foo");
});
test("leaves an already-valid AlphaDashDot name unchanged", () => {
expect(sanitizeRepoNameAlphaDashDot("valid-repo.name")).toBe("valid-repo.name");
});
});
describe("archiveGiteaRepo", () => {
const client = { url: "https://gitea.example.com", token: "test-token" };
let originalConsoleLog: typeof console.log;
let originalConsoleWarn: typeof console.warn;
let originalConsoleError: typeof console.error;
let originalConsoleDebug: typeof console.debug;
beforeEach(() => {
mockHttpGet.mockClear();
mockHttpPatch.mockClear();
mockHttpPost.mockClear();
mockHttpDelete.mockClear();
// Reset to benign defaults; individual tests override with mockImplementationOnce/mockImplementation.
mockHttpGet.mockImplementation(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementation(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
originalConsoleLog = console.log;
originalConsoleWarn = console.warn;
originalConsoleError = console.error;
originalConsoleDebug = console.debug;
console.log = mock(() => {});
console.warn = mock(() => {});
console.error = mock(() => {});
console.debug = mock(() => {});
});
afterEach(() => {
console.log = originalConsoleLog;
console.warn = originalConsoleWarn;
console.error = originalConsoleError;
console.debug = originalConsoleDebug;
});
test("mirror repo rename returns the new archived name", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result).toEqual({ archivedName: "archived-my-repo" });
// Rename PATCH + mirror-interval-disable PATCH
expect(mockHttpPatch).toHaveBeenCalledTimes(2);
const renameCall = mockHttpPatch.mock.calls[0];
expect(String(renameCall[0])).toContain("/api/v1/repos/owner/my-repo");
expect(renameCall[1]).toMatchObject({ name: "archived-my-repo" });
});
test("already-archived mirror repo returns the existing name without re-renaming", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "archived-my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "archived-my-repo");
expect(result).toEqual({ archivedName: "archived-my-repo" });
expect(mockHttpPatch).not.toHaveBeenCalled();
});
test("non-mirror repo archives natively and returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "regular-repo", mirror: false, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementationOnce(async () => ({
data: { archived: true },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "regular-repo");
expect(result).toEqual({ archivedName: null });
expect(mockHttpPatch).toHaveBeenCalledTimes(1);
expect(mockHttpPatch.mock.calls[0][1]).toMatchObject({ archived: true });
});
test("rename PATCH failure (primary and timestamped fallback both fail) returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementation(async () => {
throw new MockHttpError("Unprocessable Entity", 422, "Unprocessable Entity");
});
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result).toEqual({ archivedName: null });
// Primary rename attempt + timestamped fallback attempt, no interval-disable call
expect(mockHttpPatch).toHaveBeenCalledTimes(2);
});
test("mirror repo rename recovers via timestamped fallback after a primary conflict", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
let callCount = 0;
mockHttpPatch.mockImplementation(async (url: string, body?: any) => {
callCount++;
if (callCount === 1) {
// Primary rename attempt fails (e.g. AlphaDashDot conflict)
throw new MockHttpError("conflict", 422, "Unprocessable Entity");
}
// Fallback rename attempt and the interval-disable PATCH both succeed
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result.archivedName).toMatch(/^archived-\d{14}-my-repo$/);
expect(mockHttpPatch).toHaveBeenCalledTimes(3);
});
test("repository not found in Gitea returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: null,
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "missing-repo");
expect(result).toEqual({ archivedName: null });
expect(mockHttpPatch).not.toHaveBeenCalled();
});
});
+380 -5
View File
@@ -19,15 +19,19 @@ const mockCreatePreSyncBundleBackup = mock(() =>
let mockShouldCreatePreSyncBackup = false;
let mockShouldBlockSyncOnBackupFailure = true;
// Mock the database module
// Mock the database module. Every db.update(...).set(payload) is captured in
// dbUpdateSetCalls so tests can assert on what got written (e.g. archived
// repos keeping status "archived" after a Manual Sync).
const dbUpdateSetCalls: any[] = [];
const mockDb = {
insert: mock((table: any) => ({
values: mock((data: any) => Promise.resolve({ insertedId: "mock-id" }))
})),
update: mock(() => ({
set: mock(() => ({
where: mock(() => Promise.resolve())
}))
set: mock((data: any) => {
dbUpdateSetCalls.push(data);
return { where: mock(() => Promise.resolve()) };
})
}))
};
@@ -173,10 +177,74 @@ const mockHttpGet = mock(async (url: string, headers?: any) => {
headers: new Headers(),
};
}
// Only reachable at the "archived-{name}" candidate — the base name
// ("starred/broken-repo") deliberately falls through to the generic 404
// below, simulating a repo that archiveGiteaRepo already renamed in Gitea.
// original_url matches the test repository's GitHub source, so the
// fallback candidate's source-identity guard accepts it.
if (url.includes("/api/v1/repos/starred/archived-broken-repo")) {
return {
data: {
id: 792,
name: "archived-broken-repo",
mirror: true,
owner: { login: "starred" },
mirror_interval: "0h",
original_url: "https://github.com/user/broken-repo.git",
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
// Collision scenario: this archived mirror belongs to a DIFFERENT GitHub
// source (otheruser/collide-repo) that happens to share the base name with
// the test repository (user/collide-repo). The base name
// ("starred/collide-repo") falls through to the generic 404 below, so the
// archived-{name} fallback candidate is the only match — and its
// original_url must cause the source-identity guard to reject it.
if (url.includes("/api/v1/repos/starred/archived-collide-repo")) {
return {
data: {
id: 793,
name: "archived-collide-repo",
mirror: true,
owner: { login: "starred" },
mirror_interval: "0h",
original_url: "https://github.com/otheruser/collide-repo.git",
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
// Simulates Forgejo silently following a 301 redirect for a renamed repo:
// a GET for the STALE (pre-rename) path returns 200 with the repo's
// CURRENT identity in the response body (name differs from what was
// requested), exactly as Bun's fetch behaves after following Forgejo's
// redirect for a repo renamed from "renamed-repo" to
// "archived-renamed-repo". See #331 follow-up / canonical-identity
// adoption in syncGiteaRepoEnhanced.
if (url.includes("/api/v1/repos/starred/renamed-repo")) {
return {
data: {
id: 891,
name: "archived-renamed-repo",
mirror: true,
owner: { login: "starred" },
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
if (url.includes("/api/v1/repos/")) {
throw new MockHttpError("Not Found", 404, "Not Found");
}
// Handle org GET requests based on test context
if (url.includes("/api/v1/orgs/starred")) {
orgCheckCount++;
@@ -239,10 +307,18 @@ const mockHttpDelete = mock(async (url: string, headers?: any) => {
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
// Observable so tests can assert that the mirror-interval PATCH is (not)
// issued — e.g. archived repos must never have Gitea's periodic pulling
// re-enabled by a Manual Sync.
const mockHttpPatch = mock(async (url: string, body?: any, headers?: any) => {
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
mock.module("@/lib/http-client", () => ({
httpGet: mockHttpGet,
httpPost: mockHttpPost,
httpDelete: mockHttpDelete,
httpPatch: mockHttpPatch,
HttpError: MockHttpError
}));
@@ -284,6 +360,8 @@ describe("Enhanced Gitea Operations", () => {
mockHttpGet.mockClear();
mockHttpPost.mockClear();
mockHttpDelete.mockClear();
mockHttpPatch.mockClear();
dbUpdateSetCalls.length = 0;
mockCreatePreSyncBundleBackup.mockClear();
mockCreatePreSyncBundleBackup.mockImplementation(() =>
Promise.resolve({ bundlePath: "/tmp/mock.bundle" })
@@ -612,6 +690,303 @@ describe("Enhanced Gitea Operations", () => {
expect(String(mirrorSyncCalls[0][0])).not.toContain("/api/v1/repos/ceph/test-repo/mirror-sync");
});
test("falls back to the archived-{name} candidate when repository.status is 'archived'", async () => {
// Regression for #331 follow-up: repos archived before mirroredLocation
// was backfilled on rename (or any lingering false-positive orphan hit)
// are unreachable by name/expected-owner alone once archiveGiteaRepo has
// renamed them in Gitea to `archived-{sanitized name}`. syncGiteaRepoEnhanced
// must still find them via "Manual Sync" without manual intervention.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoArchived1",
name: "broken-repo",
fullName: "user/broken-repo",
owner: "user",
cloneUrl: "https://github.com/user/broken-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
visibility: "public",
userId: "user123",
// No mirroredLocation recorded — this repo predates the DB backfill
// added alongside archiveGiteaRepo's new return value.
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-broken-repo/mirror-sync"
);
// The base (pre-archive) name must have been probed and rejected
// (404) before falling back to the archived-{name} candidate.
const repoInfoGets = mockHttpGet.mock.calls.filter((call) =>
String(call[0]).includes("/api/v1/repos/starred/")
);
expect(
repoInfoGets.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/broken-repo")
)
).toBe(true);
expect(
repoInfoGets.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/archived-broken-repo")
)
).toBe(true);
});
test("adopts canonical identity from response body when GET follows a stale-name redirect", async () => {
// Regression for #331 follow-up, verified end-to-end on Forgejo 15.0.3:
// when a repo has been renamed (e.g. by the orphan-archive flow, or
// manually by a user), Forgejo answers a GET for the OLD name with a
// 301 redirect to the new name. Bun's fetch follows it silently and
// returns 200 with the repo's CURRENT data in the body, while the
// code still has the stale name it requested. If syncGiteaRepoEnhanced
// kept using the requested (stale) name for the follow-up POST
// .../mirror-sync, that POST would hit the same 301, get its method
// downgraded to GET per the WHATWG redirect spec, and the POST-only
// endpoint would return 405. This must work for non-archived repos
// too — a user renaming a repo in Forgejo manually is the general case.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoRenamed1",
name: "renamed-repo",
fullName: "user/renamed-repo",
owner: "user",
cloneUrl: "https://github.com/user/renamed-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("mirrored"),
visibility: "public",
userId: "user123",
// Stale: recorded before the rename happened in Gitea/Forgejo.
mirroredLocation: "starred/renamed-repo",
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-renamed-repo/mirror-sync"
);
expect(String(mirrorSyncCalls[0][0])).not.toContain(
"/api/v1/repos/starred/renamed-repo/mirror-sync"
);
});
test("keeps archived repos archived and skips the mirror-interval PATCH on Manual Sync", async () => {
// Documented contract (AutomationSettings.tsx): "Archive renames mirror
// backups with an archived- prefix and disables automatic syncs—use
// Manual Sync when you want to refresh." A successful Manual Sync of an
// archived repo must therefore refresh once WITHOUT (a) flipping status
// to "synced" (which would re-enroll it into the scheduler's auto-sync
// pool), (b) clearing the archived errorMessage annotation, or
// (c) PATCHing the mirror interval (which would re-enable Forgejo's own
// periodic pulling that archiveGiteaRepo disabled).
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
// Would normally trigger the mirror-interval PATCH on every sync.
mirrorInterval: "8h",
},
};
const repository: Repository = {
id: "repoArchived2",
name: "broken-repo",
fullName: "user/broken-repo",
owner: "user",
url: "https://github.com/user/broken-repo",
cloneUrl: "https://github.com/user/broken-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
isArchived: true,
visibility: "public",
userId: "user123",
errorMessage: "Repository archived - no longer in GitHub",
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
// The mirror-sync itself must still happen (that's the point of
// Manual Sync on an archived repo).
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-broken-repo/mirror-sync"
);
// No mirror-interval PATCH despite config.giteaConfig.mirrorInterval.
expect(mockHttpPatch).not.toHaveBeenCalled();
// The success-path DB update (the one recording lastMirrored) keeps
// status "archived" and does not clear errorMessage.
const successUpdate = dbUpdateSetCalls.find((data) => "lastMirrored" in data);
expect(successUpdate).toBeDefined();
expect(successUpdate.status).toBe("archived");
expect("errorMessage" in successUpdate).toBe(false);
expect(successUpdate.mirroredLocation).toBe("starred/archived-broken-repo");
});
test("rejects an archived-{name} fallback candidate whose original_url points at a different source", async () => {
// Two sources sharing a base name: the user mirrors user/collide-repo,
// but starred/archived-collide-repo in Gitea is the archived mirror of
// otheruser/collide-repo. The guessed archived-{name} fallback must be
// rejected via its original_url instead of syncing (and rewriting the
// DB row of) the wrong repository. With every candidate exhausted, the
// sync fails with the not-found error.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoCollide1",
name: "collide-repo",
fullName: "user/collide-repo",
owner: "user",
url: "https://github.com/user/collide-repo",
cloneUrl: "https://github.com/user/collide-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
isArchived: true,
visibility: "public",
userId: "user123",
// No mirroredLocation — forces reliance on the guessed fallback.
createdAt: new Date(),
updatedAt: new Date(),
};
await expect(
syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
)
).rejects.toThrow("Repository collide-repo not found in Gitea. Tried locations:");
// The wrong repo must never receive a mirror-sync POST.
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(0);
// The fallback candidate WAS probed (and then rejected by the guard).
expect(
mockHttpGet.mock.calls.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/archived-collide-repo")
)
).toBe(true);
});
test("blocks sync when pre-sync snapshot fails and blocking is enabled", async () => {
mockShouldCreatePreSyncBackup = true;
mockShouldBlockSyncOnBackupFailure = true;
+137 -10
View File
@@ -26,6 +26,7 @@ import {
strategyNeedsDetection,
} from "./repo-backup";
import { detectForcePush } from "./utils/force-push-detection";
import { sanitizeRepoNameAlphaDashDot } from "./gitea";
import {
parseRepositoryMetadataState,
serializeRepositoryMetadataState,
@@ -43,19 +44,45 @@ type SyncDependencies = {
/**
* Enhanced repository information including mirror status
*/
interface GiteaRepoInfo {
export interface GiteaRepoInfo {
id: number;
name: string;
owner: { login: string } | string;
mirror: boolean;
mirror_interval?: string;
clone_url?: string;
// Original migration source URL. Gitea/Forgejo populate this with the
// upstream clone address for migrated/mirrored repos, so it is the
// authoritative way to tell whether an existing mirror points at THIS
// GitHub source (vs. a same-named mirror of a different source).
original_url?: string;
private: boolean;
}
interface SyncTargetCandidate {
owner: string;
repoName: string;
/**
* True for the guessed `archived-{name}` fallback candidate (see
* syncGiteaRepoEnhanced). Unlike the recorded mirroredLocation or the
* expected-owner candidate, this one is derived purely from the repo NAME,
* so it can collide with a different source repo that shares the same base
* name it must pass an original_url source check before being accepted.
*/
isArchivedFallback?: boolean;
}
/**
* Normalize a git remote URL for source-identity comparison: lowercase,
* strip trailing slashes and a trailing `.git`.
*/
function normalizeSourceUrl(url: string): string {
return url
.trim()
.toLowerCase()
.replace(/\/+$/, "")
.replace(/\.git$/, "")
.replace(/\/+$/, "");
}
function parseMirroredLocation(location?: string | null): SyncTargetCandidate | null {
@@ -324,12 +351,32 @@ export async function syncGiteaRepoEnhanced({
// Resolve sync target in a backward-compatible order:
// 1) recorded mirroredLocation (actual historical mirror location)
// 2) owner derived from current strategy/config
// 3) (archived repos only) the `archived-{name}` rename that
// archiveGiteaRepo applies to mirror repos (see #331 follow-up).
// Repos archived before mirroredLocation was backfilled on rename, or
// any lingering false-positive orphan hit, would otherwise be
// unreachable from "Manual Sync" — the UI's documented way to refresh
// an archived mirror — because the recorded/expected name no longer
// exists and Gitea returns HTTP 405 for it.
const dependencies = deps ?? (await import("./gitea"));
const expectedOwner = await dependencies.getGiteaRepoOwnerAsync({ config, repository });
const recordedTarget = parseMirroredLocation(repository.mirroredLocation);
// Archived state can live in either field: the cleanup service sets both,
// but a failed retry can clobber `status` while `isArchived` survives.
const isArchivedRepo =
repository.status === "archived" || !!repository.isArchived;
const candidateTargets = dedupeSyncTargets([
...(recordedTarget ? [recordedTarget] : []),
{ owner: expectedOwner, repoName: repository.name },
...(isArchivedRepo
? [
{
owner: expectedOwner,
repoName: `archived-${sanitizeRepoNameAlphaDashDot(repository.name)}`,
isArchivedFallback: true,
},
]
: []),
]);
let repoOwner = expectedOwner;
@@ -355,8 +402,45 @@ export async function syncGiteaRepoEnhanced({
continue;
}
repoOwner = target.owner;
repoName = target.repoName;
// The archived-{name} fallback candidate is guessed from the repo NAME
// alone, so it can hit a DIFFERENT source's archived mirror when two
// sources share a base name (e.g. foo/tools and bar/tools both mirrored;
// foo's archived as `archived-tools`). Before accepting it, verify the
// candidate's original_url (the authoritative migration source — see
// GiteaRepoInfo) points at THIS repository's GitHub source. An empty/
// unset original_url is accepted as before (some migrations leave it
// unset). This guard deliberately does NOT apply to the recorded
// mirroredLocation or expected-name candidates.
if (
target.isArchivedFallback &&
typeof candidateInfo.original_url === "string" &&
candidateInfo.original_url.trim() !== ""
) {
const candidateSource = normalizeSourceUrl(candidateInfo.original_url);
const ownSources = [repository.cloneUrl, repository.url]
.filter((u): u is string => typeof u === "string" && u.trim() !== "")
.map(normalizeSourceUrl);
if (!ownSources.includes(candidateSource)) {
console.warn(
`[Sync] Skipping archived-name candidate ${target.owner}/${target.repoName} for ${repository.name}: its original_url (${candidateInfo.original_url}) points at a different source`
);
continue;
}
}
// Adopt the canonical identity from the response, not the requested target.
// Gitea/Forgejo answer GETs for a renamed repo's old name with a 301 that
// fetch follows silently, so `target` may be a stale pre-rename name; a
// follow-up POST (mirror-sync) to the stale path gets its method downgraded
// by the redirect and fails with 405. The response body always carries the
// repo's current name/owner (#331 follow-up, verified on Forgejo 15).
const canonicalOwner =
typeof candidateInfo.owner === "string"
? candidateInfo.owner
: candidateInfo.owner?.login;
repoOwner = canonicalOwner || target.owner;
repoName = candidateInfo.name || target.repoName;
repoInfo = candidateInfo;
break;
}
@@ -544,9 +628,12 @@ export async function syncGiteaRepoEnhanced({
// Create backup if strategy says so
if (shouldBackupForStrategy(backupStrategy, forcePushDetected)) {
const cloneUrl =
repoInfo.clone_url ||
`${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
// Always derive the clone URL from the user-configured Gitea URL rather
// than repoInfo.clone_url (which reflects Gitea's ROOT_URL and may be
// unreachable from the app — e.g. Tailscale MagicDNS deployments where
// ROOT_URL resolves externally but the app talks to Gitea on a private
// address).
const cloneUrl = `${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
try {
const backupResult = await createPreSyncBundleBackup({
@@ -613,7 +700,13 @@ export async function syncGiteaRepoEnhanced({
// NOTE: Gitea/Forgejo's PATCH /repos/{owner}/{repo} API does not support
// updating mirror credentials (mirror_username/mirror_password). Repos that
// were originally migrated without credentials must be deleted and re-mirrored.
if (config.giteaConfig?.mirrorInterval) {
//
// Skipped for archived repos: archiveGiteaRepo deliberately disabled
// Gitea's own periodic pulling, and the documented contract
// (AutomationSettings.tsx: "Archive ... disables automatic syncs—use
// Manual Sync when you want to refresh") says a Manual Sync refreshes
// once without re-enabling any automatic syncing.
if (config.giteaConfig?.mirrorInterval && !isArchivedRepo) {
try {
console.log(`[Sync] Updating mirror interval for ${repoOwner}/${repoName} to ${config.giteaConfig.mirrorInterval}`);
const updateUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}`;
@@ -836,14 +929,22 @@ export async function syncGiteaRepoEnhanced({
metadataState.lastSyncedAt = new Date().toISOString();
}
// Mark repo as "synced" in DB
// Mark repo as "synced" in DB — unless it's archived. The documented
// contract (AutomationSettings.tsx: "Archive ... disables automatic
// syncs—use Manual Sync when you want to refresh") means a Manual Sync
// of an archived repo must NOT re-enroll it into the scheduler's
// auto-sync pool (which selects mirrored/synced/failed/pending) nor
// clear the archived errorMessage annotation shown in the UI; it still
// records lastMirrored and the (possibly corrected) mirroredLocation.
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("synced"),
status: isArchivedRepo
? repoStatusEnum.parse("archived")
: repoStatusEnum.parse("synced"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
...(isArchivedRepo ? {} : { errorMessage: null }),
mirroredLocation: `${repoOwner}/${repoName}`,
metadata: metadataUpdated
? serializeRepositoryMetadataState(metadataState)
@@ -886,6 +987,32 @@ export async function syncGiteaRepoEnhanced({
status: "failed",
});
}
} else if (syncError instanceof HttpError && syncError.status === 405) {
// Gitea returns HTTP 405 (with an empty body) when the repository is not
// a pull-mirror in its database — e.g. Gitea auto-disabled the mirror or
// the repo lost its mirror state after a manual edit.
const actionableMessage =
`Gitea reports this repository is not a pull mirror (HTTP 405). ` +
`In Gitea check Settings → Mirror Settings; if the mirror section is ` +
`missing, delete the repository in Gitea and re-mirror it from gitea-mirror.`;
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("failed"),
updatedAt: new Date(),
errorMessage: actionableMessage,
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Sync failed: ${repository.name} is not a pull mirror in Gitea (HTTP 405)`,
details: actionableMessage,
status: "failed",
});
}
throw syncError;
}
+52
View File
@@ -0,0 +1,52 @@
/**
* Regression test for the #334 sibling bug labels silently dropped on issue update.
*
* Gitea/Forgejo's `EditIssueOption` has no `labels` field (only `CreateIssueOption`
* does), so a `labels` key in a `PATCH .../issues/{index}` body is silently ignored.
* The old code put `labels` in the update PATCH, so label changes never propagated
* onto already-mirrored issues. The fix builds the edit body WITHOUT labels
* (`buildGiteaIssueEditPayload`) and reconciles labels separately through the
* sub-resource `PUT .../issues/{index}/labels` (`buildGiteaIssueLabelsPayload`).
*
* Verified live against Gitea 1.24.7: PATCH with `labels` leaves the issue's labels
* unchanged; PUT to the labels sub-resource replaces them. Confirmed end-to-end that
* a drifted (label-less) mirrored issue reconciles back to its GitHub label set.
*/
import { describe, test, expect } from "bun:test";
import { buildGiteaIssueEditPayload, buildGiteaIssueLabelsPayload } from "@/lib/gitea";
describe("buildGiteaIssueEditPayload (#334 sibling)", () => {
test("edit body never carries `labels` (Gitea's EditIssueOption ignores it)", () => {
const payload = buildGiteaIssueEditPayload({
title: "[GH-ISSUE #1] Fix the thing",
body: "desc",
closed: false,
});
expect(payload).not.toHaveProperty("labels");
expect(payload).toEqual({
title: "[GH-ISSUE #1] Fix the thing",
body: "desc",
state: "open",
});
});
test("maps the closed flag to Gitea's `state`", () => {
expect(buildGiteaIssueEditPayload({ title: "t", body: "b", closed: true }).state).toBe("closed");
expect(buildGiteaIssueEditPayload({ title: "t", body: "b", closed: false }).state).toBe("open");
});
});
describe("buildGiteaIssueLabelsPayload (#334 sibling)", () => {
test("replaces the full label set with the resolved Gitea label ids", () => {
expect(buildGiteaIssueLabelsPayload([7, 9])).toEqual({ labels: [7, 9] });
});
test("sends an empty set so upstream label removals propagate", () => {
expect(buildGiteaIssueLabelsPayload([])).toEqual({ labels: [] });
});
test("treats a missing id list as an empty set (defensive)", () => {
expect(buildGiteaIssueLabelsPayload(undefined as any)).toEqual({ labels: [] });
});
});
+51
View File
@@ -0,0 +1,51 @@
/**
* Regression test for #334 "Release titles not being mirrored properly".
*
* Root cause: the release create/update payloads sent the release title under the
* JSON key `title`, but Gitea/Forgejo's release API expects `name` (the API Go
* struct is `Title string \`json:"name"\``). `title` is silently dropped, so every
* mirrored release landed with a blank name.
*
* `buildGiteaReleasePayload` is the single source of truth for both the create
* (POST) and update (PATCH) bodies. Verified live against Gitea 1.24.7: a payload
* with `title` yields `name: ""`; a payload with `name` sets the title correctly.
*/
import { describe, test, expect } from "bun:test";
import { buildGiteaReleasePayload } from "@/lib/gitea";
describe("buildGiteaReleasePayload (#334)", () => {
test("carries the release title under `name`, never `title`", () => {
const payload = buildGiteaReleasePayload(
{ tag_name: "v0.19.0", name: "v0.19.0", draft: false, prerelease: false },
"## Features\n- something"
);
expect(payload.name).toBe("v0.19.0");
expect(payload).not.toHaveProperty("title");
expect(payload).toEqual({
tag_name: "v0.19.0",
name: "v0.19.0",
body: "## Features\n- something",
draft: false,
prerelease: false,
});
});
test("falls back to tag_name when the GitHub release name is empty or null", () => {
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3", name: null }, "x").name).toBe("v1.2.3");
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3", name: "" }, "x").name).toBe("v1.2.3");
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3" }, "x").name).toBe("v1.2.3");
});
test("passes draft/prerelease/body through unchanged", () => {
const payload = buildGiteaReleasePayload(
{ tag_name: "v2.0.0", name: "Two", draft: true, prerelease: true },
"notes body"
);
expect(payload.body).toBe("notes body");
expect(payload.draft).toBe(true);
expect(payload.prerelease).toBe(true);
expect(payload.tag_name).toBe("v2.0.0");
});
});
+232
View File
@@ -0,0 +1,232 @@
/**
* Unit tests for release reconciliation logic regression for #310.
*
* Root-cause verdict: Theory A (not Theory B).
*
* The old `needsRecreation` check compared published_at-based expected indices
* against Gitea's API order (which mirrors sort by tag-commit-date, not
* published_at). For repos where published_at order permanently disagrees with
* tag-commit-date order, `currentExpectedIdx < nextExpectedIdx` evaluates true
* on every single sync, triggering delete-and-recreate forever.
*
* Discriminating evidence unaconfig_dart fixture:
* v0.1.1 published_at 2024-01-13T23:44 (earlier) expectedOrder index 0
* v0.1.0 published_at 2024-01-16T00:23 (later) expectedOrder index 1
* v0.1.0 tagged 2024-01-13T23:30 (earlier tag commit)
* v0.1.1 tagged 2024-01-13T23:42 (later tag commit)
* Gitea tag-commit order: [v0.1.1, v0.1.0] (v0.1.1 has newer commit)
* Old check: currentExpectedIdx(v0.1.1)=0 < nextExpectedIdx(v0.1.0)=1 TRUE
* needsRecreation fires on every sync, forever.
*
* Theory B (operator inversion) would fire for ALL repos with >1 release, but
* field evidence shows only ~2 of 150 repos are affected ruling it out.
*
* Fix: replaced `needsRecreation` machinery with set-based reconciliation via
* `classifyReleasesForReconciliation`. Releases are created when missing, skipped
* (or PATCH-updated if content drifted) when present. No deletions for ordering.
*/
import { describe, expect, it } from "bun:test";
import {
classifyReleasesForReconciliation,
classifyAssetsForReconciliation,
} from "@/lib/gitea";
describe("classifyReleasesForReconciliation", () => {
describe("normal repo — published_at order matches tag-commit order", () => {
it("creates releases missing in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
expect(toSkip).toEqual([]);
});
it("skips releases already present in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.1.0", "v1.2.0"];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
});
it("creates missing releases while skipping existing ones", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.2.0"]; // v1.1.0 is missing
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.1.0"]);
expect(toSkip).toEqual(["v1.0.0", "v1.2.0"]);
});
it("does NOT produce any deletions — order mismatches are ignored", () => {
// Even if Gitea has them in a different order, the function never suggests deletion
const github = ["v1.0.0", "v1.1.0"];
const gitea = ["v1.1.0", "v1.0.0"]; // reversed order from Gitea
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v1.0.0");
expect(toSkip).toContain("v1.1.0");
});
});
describe("unaconfig_dart regression — published_at order disagrees with tag-commit order (#310)", () => {
// v0.1.0: tagged 2024-01-13T23:30, published_at 2024-01-16T00:23 (published AFTER v0.1.1)
// v0.1.1: tagged 2024-01-13T23:42, published_at 2024-01-13T23:44 (published BEFORE v0.1.0)
//
// Gitea display order (by tag-commit date): [v0.1.1, v0.1.0] (v0.1.1 tagged later)
// GitHub published_at order (oldest first): [v0.1.1, v0.1.0] (v0.1.1 published earlier)
// Wait — in this specific case the orders AGREE. The inversion scenario is:
// GitHub sorts descending (newest first): [v0.1.0, v0.1.1]
// Gitea API returns by tag-commit DESC: [v0.1.1, v0.1.0]
// Old expectedOrder (ascending published): v0.1.1→0, v0.1.0→1
// Check for [v0.1.1, v0.1.0]: current=v0.1.1(idx=0) < next=v0.1.0(idx=1) → TRUE every time
it("does NOT trigger recreation when published_at order and tag-commit order disagree", () => {
// Both releases already in Gitea (as they would be after first successful sync).
// Old code would fire needsRecreation=true here on every subsequent sync.
// New code: set-based check — both present → toCreate is empty → no deletions.
const github = ["v0.1.0", "v0.1.1"]; // GitHub API returns newest published_at first
const gitea = ["v0.1.1", "v0.1.0"]; // Gitea tag-commit order (v0.1.1 tagged later)
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]); // nothing to create
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v0.1.0");
expect(toSkip).toContain("v0.1.1");
});
it("creates v0.1.0 and v0.1.1 when Gitea has no releases yet (first sync)", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0", "v0.1.1"]);
expect(toSkip).toEqual([]);
});
it("only creates the missing release when one of the two already exists", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea = ["v0.1.1"]; // only v0.1.1 was created so far
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0"]);
expect(toSkip).toEqual(["v0.1.1"]);
});
});
describe("edge cases", () => {
it("handles empty GitHub releases list", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], ["v1.0.0"]);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("handles both lists empty", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], []);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("ignores Gitea releases that are not in the GitHub set (orphans, handled by retention cleanup)", () => {
const github = ["v1.0.0"];
const gitea = ["v1.0.0", "v0.9.0"]; // v0.9.0 is an orphan not in GitHub's limited set
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0"]);
// v0.9.0 not mentioned in either output — handled by retention cleanup, not here
});
});
});
/**
* Asset reconciliation regression for #331.
*
* Root cause: assets were uploaded only on the create path. When a Gitea release
* already existed (every re-sync, or after an interrupted first upload), the update
* path PATCHed the body and `continue`d without ever touching assets so a release
* that existed without its full asset set stayed permanently asset-less and re-syncing
* could never heal it. Reproduced on a real Forgejo pull-mirror: GitHub release with
* two 35-40MB binaries Gitea release with 0 assets re-sync logged "Updating
* existing release" and left it at 0.
*
* Fix: reconcile assets idempotently on both paths via classifyAssetsForReconciliation.
*/
describe("classifyAssetsForReconciliation", () => {
it("uploads all assets when the Gitea release has none (the #331 broken state)", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea: Array<{ id: number; name: string; size: number }> = [];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([
{ name: "base.zip", replaceAssetId: null },
{ name: "extras.zip", replaceAssetId: null },
]);
});
it("backfills only the missing asset when one already exists", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [{ id: 9, name: "base.zip", size: 40_264_954 }];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual(["base.zip"]);
expect(toUpload).toEqual([{ name: "extras.zip", replaceAssetId: null }]);
});
it("is idempotent — skips everything when all assets already match by name+size", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [
{ id: 9, name: "base.zip", size: 40_264_954 },
{ id: 10, name: "extras.zip", size: 37_098_528 },
];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual(["base.zip", "extras.zip"]);
});
it("replaces an asset whose size changed upstream (re-upload over the stale copy)", () => {
const github = [{ name: "firmware.bin", size: 2048 }];
const gitea = [{ id: 42, name: "firmware.bin", size: 1024 }]; // truncated/stale
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([{ name: "firmware.bin", replaceAssetId: 42 }]);
});
it("handles a release with no GitHub assets", () => {
const { toUpload, toSkip } = classifyAssetsForReconciliation(
[],
[{ id: 1, name: "leftover.zip", size: 10 }]
);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual([]);
});
});
+704 -291
View File
File diff suppressed because it is too large Load Diff
+164 -9
View File
@@ -1,15 +1,23 @@
import { describe, expect, test, mock } from "bun:test";
import { getGithubRepositories } from "@/lib/github";
function makeRepo() {
function makeRepo(overrides: Partial<{
name: string;
full_name: string;
ownerLogin: string;
ownerType: string;
fork: boolean;
}> = {}) {
const ownerLogin = overrides.ownerLogin ?? "octo";
const ownerType = overrides.ownerType ?? "User";
return {
name: "demo",
full_name: "octo/demo",
html_url: "https://github.com/octo/demo",
clone_url: "https://github.com/octo/demo.git",
owner: { login: "octo", type: "User" },
name: overrides.name ?? "demo",
full_name: overrides.full_name ?? `${ownerLogin}/${overrides.name ?? "demo"}`,
html_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}`,
clone_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}.git`,
owner: { login: ownerLogin, type: ownerType },
private: false,
fork: false,
fork: overrides.fork ?? false,
has_issues: true,
archived: false,
size: 1,
@@ -23,11 +31,11 @@ function makeRepo() {
};
}
function makeOctokit() {
function makeOctokit(reposToReturn?: ReturnType<typeof makeRepo>[]) {
let captured: Record<string, unknown> | null = null;
const paginate = mock(async (_method: unknown, options?: Record<string, unknown>) => {
captured = options ?? null;
return [makeRepo()];
return reposToReturn ?? [makeRepo()];
});
return {
octokit: {
@@ -98,3 +106,150 @@ describe("getGithubRepositories - affiliation", () => {
}
});
});
describe("getGithubRepositories - skipPersonalRepos", () => {
const personalRepo = makeRepo({ name: "my-lib", ownerLogin: "octo", ownerType: "User" });
const orgRepo = makeRepo({ name: "org-lib", ownerLogin: "my-org", ownerType: "Organization" });
const otherUserRepo = makeRepo({ name: "collab-lib", ownerLogin: "other-user", ownerType: "User" });
test("default false — keeps all repos including personal", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: false } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — drops repos owned by authenticated user", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — keeps repos owned by other users (collaborator repos)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo, otherUserRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
expect(repos.map((r) => r.name)).toContain("collab-lib");
});
test("skipPersonalRepos=true with no owner configured — keeps all repos (safe fallback)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "", skipPersonalRepos: true } as any },
});
// Empty owner means we can't identify the user, so nothing should be dropped
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — unset (undefined) behaves like false", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo" } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
});
describe("getGithubRepositories - includeOrganizations allowlist", () => {
const personalRepo = makeRepo({ name: "my-lib", ownerLogin: "octo", ownerType: "User" });
const wantedOrgRepo = makeRepo({ name: "wanted", ownerLogin: "wanted-org", ownerType: "Organization" });
const otherOrgRepo = makeRepo({ name: "noise", ownerLogin: "noise-org", ownerType: "Organization" });
const collabRepo = makeRepo({ name: "collab", ownerLogin: "other-user", ownerType: "User" });
test("empty allowlist — keeps repos from all orgs (backward compat)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: [] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
test("non-empty allowlist — keeps only listed orgs, drops other orgs", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("allowlist match is case-insensitive", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["Wanted-Org"] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("allowlist never restricts personal or collaborator repos", async () => {
const { octokit } = makeOctokit([personalRepo, collabRepo, wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
});
// User-owned and collaborator repos pass through regardless of the allowlist
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("collab");
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("composes with skipPersonalRepos — drops personal, keeps only listed org", async () => {
const { octokit } = makeOctokit([personalRepo, wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: {
githubConfig: {
owner: "octo",
skipPersonalRepos: true,
includeOrganizations: ["wanted-org"],
} as any,
},
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("blank/whitespace entries are ignored (treated as empty allowlist)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: [" ", ""] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
test("includeAllOrgsOverride bypasses allowlist (cleanup safety)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
includeAllOrgsOverride: true,
});
// Override returns all org repos so cleanup never false-orphans excluded orgs
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
});
+40 -2
View File
@@ -236,6 +236,7 @@ export async function getGithubRepositories({
octokit,
config,
includeCollaboratorReposOverride,
includeAllOrgsOverride,
}: {
octokit: Octokit;
config: Partial<Config>;
@@ -243,6 +244,10 @@ export async function getGithubRepositories({
// cleanup service so we never mark a collab repo as orphaned just because
// the import filter is currently off.
includeCollaboratorReposOverride?: boolean;
// Bypass the includeOrganizations allowlist so all org repos are returned.
// Used by the cleanup service so a previously-mirrored org repo isn't flagged
// as orphaned just because the user narrowed the allowlist.
includeAllOrgsOverride?: boolean;
}): Promise<GitRepo[]> {
try {
const includeCollab =
@@ -263,10 +268,37 @@ export async function getGithubRepositories({
);
const skipForks = config.githubConfig?.skipForks ?? false;
const skipPersonalRepos = config.githubConfig?.skipPersonalRepos ?? false;
// The authenticated user's login — used to identify personally-owned repos
const authenticatedUserLogin = config.githubConfig?.owner ?? "";
// Opt-in organization allowlist. When non-empty, only repos owned by the
// listed organizations are imported; org repos from any other org the user
// happens to be a member of are dropped. Empty = import all org repos
// (backward-compatible default). Owned/collaborator repos are unaffected.
const includeOrgs = includeAllOrgsOverride
? []
: config.githubConfig?.includeOrganizations ?? [];
const allowedOrgs = new Set(
includeOrgs.map((org) => org.trim().toLowerCase()).filter(Boolean),
);
const filteredRepos = repos.filter((repo) => {
const isForkAllowed = !skipForks || !repo.fork;
return isForkAllowed;
// When skipPersonalRepos is true, drop repos owned by the authenticated user
// (owner.type === "User" and owner.login matches the configured GitHub username).
// Org repos have owner.type === "Organization" so they are always kept.
const isPersonalRepo =
skipPersonalRepos &&
authenticatedUserLogin.length > 0 &&
repo.owner.login === authenticatedUserLogin &&
repo.owner.type === "User";
// When an allowlist is configured, only keep org repos whose owning org
// is listed. Non-org repos (owned/collaborator) are never restricted here.
const isOrgAllowed =
allowedOrgs.size === 0 ||
repo.owner.type !== "Organization" ||
allowedOrgs.has(repo.owner.login.toLowerCase());
return isForkAllowed && !isPersonalRepo && isOrgAllowed;
});
return filteredRepos.map((repo) => ({
@@ -667,9 +699,11 @@ export async function getGithubStarredListNames({
export async function getGithubOrganizations({
octokit,
config,
skipOrgNames,
}: {
octokit: Octokit;
config: Partial<Config>;
skipOrgNames?: Set<string>;
}): Promise<{ organizations: GitOrg[]; failedOrgs: { name: string; avatarUrl: string; reason: string }[] }> {
try {
const { data: orgs } = await octokit.orgs.listForAuthenticatedUser({
@@ -682,7 +716,7 @@ export async function getGithubOrganizations({
? excludedOrgsEnv.split(",").map((org) => org.trim().toLowerCase())
: [];
// Filter out excluded organizations
// Filter out excluded and user-ignored organizations
const filteredOrgs = orgs.filter((org) => {
if (excludedOrgs.includes(org.login.toLowerCase())) {
console.log(
@@ -690,6 +724,10 @@ export async function getGithubOrganizations({
);
return false;
}
if (skipOrgNames?.has(org.login.toLowerCase())) {
console.log(`Skipping organization ${org.login} - ignored by user`);
return false;
}
return true;
});
@@ -0,0 +1,52 @@
/**
* Unit tests for the pure orphan-verdict decision logic regression coverage
* for issue #331's root cause: `identifyOrphanedRepositories()` treated a DB
* repository as "orphaned" the moment it was missing from a single bulk
* GitHub fetch (owned+collaborator+org repos, plus starred repos). That bulk
* fetch can be transiently incomplete (rate-limit timing, GraphQL star-list
* pagination quirks, org-allowlist edge cases, etc.), producing false
* positives that got archived (renamed to `archived-{name}` in Gitea/Forgejo)
* even though the repo was never actually removed/unstarred on GitHub.
*
* The fix adds a second, targeted confirmation call for any repo that merely
* *looks* orphaned from the bulk list before finalizing it as such.
* `resolveOrphanVerdict` is the pure decision function extracted from that
* flow (similar in spirit to classifyAssetsForReconciliation /
* classifyReleasesForReconciliation in gitea-releases.test.ts) so the
* decision logic itself is unit-testable without hitting the DB or octokit.
*/
import { describe, test, expect } from "bun:test";
import { resolveOrphanVerdict } from "./repository-cleanup-service";
describe("resolveOrphanVerdict", () => {
test("repo present in the bulk fetch is never orphaned, regardless of the direct check", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: true, directCheckConfirmsGone: true })
).toBe(false);
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: true, directCheckConfirmsGone: false })
).toBe(false);
});
test("repo missing from the bulk fetch is orphaned only when the direct check confirms it's gone (404)", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: true })
).toBe(true);
});
test("repo missing from the bulk fetch but still found by the direct check is NOT orphaned (bulk fetch was incomplete)", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: false })
).toBe(false);
});
test("repo missing from the bulk fetch whose direct check itself failed (network error, rate limit, etc.) fails safe as NOT orphaned", () => {
// directCheckConfirmsGone is only true for a clean, explicit 404 — any
// other outcome (including a failed verification call) is represented
// as false by the caller, which must resolve to "not orphaned" here.
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: false })
).toBe(false);
});
});
+154 -14
View File
@@ -15,6 +15,37 @@ import { isMirrorableGitHubRepo } from '@/lib/repo-eligibility';
let cleanupInterval: NodeJS.Timeout | null = null;
let isCleanupRunning = false;
/**
* Decide whether a DB repository that appears to be missing from the bulk
* GitHub fetch should actually be treated as orphaned.
*
* The bulk fetch (owned+collaborator+org repos, plus starred repos) that
* feeds `fullNameFoundInBulkList` can be transiently incomplete rate-limit
* timing, GraphQL star-list pagination quirks, org-allowlist edge cases,
* etc. so a repo missing from it is only a *candidate*, not a confirmed
* orphan. It is only orphaned when a direct, targeted GitHub call ALSO
* confirms the repo is gone (a clean 404). Any other outcome the repo
* still exists, or the direct check itself failed for some other reason
* (network error, rate limit, 5xx, timeout) must NOT be treated as
* orphaned; this fails safe and matches the existing fail-safe philosophy
* already in this module for GitHub API errors.
*
* Pure/exported so the decision logic is unit-testable without hitting the
* DB or octokit (see repository-cleanup-service.test.ts).
*/
export function resolveOrphanVerdict({
fullNameFoundInBulkList,
directCheckConfirmsGone,
}: {
fullNameFoundInBulkList: boolean;
directCheckConfirmsGone: boolean;
}): boolean {
if (fullNameFoundInBulkList) {
return false;
}
return directCheckConfirmsGone;
}
/**
* Identify orphaned repositories for a user
* These are repositories that exist in our database (and likely in Gitea)
@@ -33,12 +64,18 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
let githubApiAccessible = true;
try {
// Fetch GitHub data. Always include collaborator repos here regardless
// of the user's import filter, otherwise repos previously mirrored as a
// collaborator would be flagged as orphaned and archived/deleted as soon
// as the user disables the filter.
// Fetch GitHub data. Always include collaborator repos and bypass the
// organization allowlist here regardless of the user's import filters,
// otherwise repos previously mirrored as a collaborator or from an org the
// user later removed from the allowlist would be flagged as orphaned and
// archived/deleted as soon as the user narrows those filters.
const [basicAndForkedRepos, starredRepos] = await Promise.all([
getGithubRepositories({ octokit, config, includeCollaboratorReposOverride: true }),
getGithubRepositories({
octokit,
config,
includeCollaboratorReposOverride: true,
includeAllOrgsOverride: true,
}),
config.githubConfig?.includeStarred
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
@@ -74,8 +111,12 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
.where(eq(repositories.userId, userId));
// Only identify repositories as orphaned if we successfully accessed GitHub
// This prevents false positives when GitHub is down or account is inaccessible
const orphanedRepos = dbRepos.filter(repo => {
// This prevents false positives when GitHub is down or account is inaccessible.
//
// First pass (sync, cheap): filter down to repos that merely *look*
// orphaned based on map membership against the single bulk fetch above.
// This is the false-positive-prone signal — see resolveOrphanVerdict.
const candidateOrphans = dbRepos.filter(repo => {
// Skip repositories we've already archived/preserved
if (repo.status === 'archived' || repo.isArchived) {
console.log(`[Repository Cleanup] Skipping ${repo.fullName} - already archived`);
@@ -91,6 +132,8 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
const githubRepo = githubReposByFullName.get(repo.fullName);
if (!githubRepo) {
// Missing from the bulk list — candidate for direct confirmation below,
// not yet a confirmed orphan.
return true;
}
@@ -101,11 +144,93 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
return false;
});
if (candidateOrphans.length === 0) {
return [];
}
// Second pass (async, targeted): confirm each candidate directly against
// GitHub before finalizing it as orphaned. This only adds extra API calls
// for the (presumably small) set of repos that look orphaned, not for
// every repo, so it shouldn't meaningfully increase rate-limit pressure
// in the common case (few or no orphans per run). Promise.allSettled so
// one repo's verification failure can't block the others.
const verificationOutcomes = await Promise.allSettled(
candidateOrphans.map(async (repo) => {
if (repo.isStarred) {
try {
await octokit.rest.activity.checkRepoIsStarredByAuthenticatedUser({
owner: repo.owner,
repo: repo.name,
});
// Resolves (no throw) => still starred; the bulk star fetch
// missed it. Fail safe: do not treat as orphaned.
return { repo, directCheckConfirmsGone: false };
} catch (starError: any) {
if (starError?.status === 404) {
return { repo, directCheckConfirmsGone: true };
}
console.warn(
`[Repository Cleanup] Direct star-check for ${repo.fullName} failed with a non-404 error; skipping this cycle to be safe: ${
starError instanceof Error ? starError.message : String(starError)
}`
);
return { repo, directCheckConfirmsGone: false };
}
}
try {
await octokit.rest.repos.get({ owner: repo.owner, repo: repo.name });
// Resolves (no throw) => repo still exists; the bulk fetch missed
// it (e.g. an org-allowlist edge case). Fail safe: not orphaned.
return { repo, directCheckConfirmsGone: false };
} catch (repoError: any) {
if (repoError?.status === 404) {
return { repo, directCheckConfirmsGone: true };
}
console.warn(
`[Repository Cleanup] Direct existence check for ${repo.fullName} failed with a non-404 error; skipping this cycle to be safe: ${
repoError instanceof Error ? repoError.message : String(repoError)
}`
);
return { repo, directCheckConfirmsGone: false };
}
})
);
const orphanedRepos = verificationOutcomes
.map((outcome, index) => {
if (outcome.status !== 'fulfilled') {
const repo = candidateOrphans[index];
console.warn(
`[Repository Cleanup] Direct orphan verification threw unexpectedly for ${repo.fullName}; skipping this cycle to be safe: ${
outcome.reason instanceof Error ? outcome.reason.message : String(outcome.reason)
}`
);
return null;
}
const { repo, directCheckConfirmsGone } = outcome.value;
const isOrphaned = resolveOrphanVerdict({
fullNameFoundInBulkList: false,
directCheckConfirmsGone,
});
if (!isOrphaned) {
return null;
}
console.log(
`[Repository Cleanup] Confirmed orphaned via direct GitHub check: ${repo.fullName}`
);
return repo;
})
.filter((repo): repo is (typeof dbRepos)[number] => repo !== null);
if (orphanedRepos.length > 0) {
console.log(`[Repository Cleanup] Found ${orphanedRepos.length} orphaned repositories for user ${userId}`);
}
return orphanedRepos;
} catch (error) {
console.error(`[Repository Cleanup] Error identifying orphaned repositories for user ${userId}:`, error);
@@ -179,15 +304,30 @@ async function handleOrphanedRepository(
// Non-fatal; continue with best guess
}
await archiveGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
// Update database status
await db.update(repositories).set({
const { archivedName } = await archiveGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
// Update database status. If the archive call renamed the repo in Gitea
// (mirror path), persist the new location so a subsequent "Manual Sync"
// (the UI's documented path for refreshing an archived mirror) can find
// it by its actual current name instead of the stale pre-rename one —
// otherwise syncGiteaRepoEnhanced looks up a name that no longer exists
// and Gitea returns HTTP 405 ("not a pull mirror").
//
// Only mirroredLocation gets the Gitea-side `archived-{name}` value.
// Do NOT write it into `name`: repositories.name is consumed as the
// GITHUB repo name elsewhere (release listing, force-push detection),
// and mirroredLocation alone is what syncGiteaRepoEnhanced resolves
// first when locating the Gitea mirror.
const dbUpdate: Record<string, any> = {
status: 'archived',
isArchived: true,
errorMessage: 'Repository archived - no longer in GitHub',
updatedAt: new Date(),
}).where(eq(repositories.id, repo.id));
};
if (archivedName && archivedName !== giteaRepoName) {
dbUpdate.mirroredLocation = `${giteaOwner}/${archivedName}`;
}
await db.update(repositories).set(dbUpdate).where(eq(repositories.id, repo.id));
// Create event
await publishEvent({
+22
View File
@@ -103,6 +103,28 @@ describe("Scheduler Service - Ignored Repository Handling", () => {
]);
});
test("auto-start gate: enabled=true → should start, enabled=false → should not start even with mirrorInterval", () => {
// Mirror the gate logic from checkAutoStartConfiguration / performInitialAutoStart.
// The enabled flag is the single authoritative signal; a configured
// mirrorInterval is a timing detail and must not bypass a disabled toggle.
const shouldAutoStart = (scheduleConfig?: { enabled?: boolean }) =>
scheduleConfig?.enabled === true;
expect(shouldAutoStart({ enabled: true })).toBe(true);
expect(shouldAutoStart({ enabled: false })).toBe(false);
expect(shouldAutoStart({})).toBe(false);
expect(shouldAutoStart(undefined)).toBe(false);
// Simulating: user disabled scheduling but has a mirrorInterval configured.
// The old code checked `scheduleEnabled || hasMirrorInterval`; the fix
// ensures only the enabled flag is checked.
const configWithIntervalButDisabled = {
scheduleConfig: { enabled: false },
giteaConfig: { mirrorInterval: "8h" },
};
expect(shouldAutoStart(configWithIntervalButDisabled.scheduleConfig)).toBe(false);
});
test("should validate all repository status enum values", () => {
const validStatuses = [
"imported",
+36 -10
View File
@@ -266,6 +266,27 @@ async function runScheduledSync(config: any): Promise<void> {
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
// A `failed` repo whose recorded location still resolves to a
// live same-source mirror (e.g. migrate succeeded but metadata
// failed) must be SYNCED, not re-created — otherwise the
// re-create loop spawns suffixed duplicates (#315). The create
// path also reuses now, but routing to sync here avoids a
// wasted migrate attempt and keeps recovery cheap.
if (repo.status === 'failed' && repository.mirroredLocation) {
const { findExistingMirror } = await import('@/lib/utils/mirror-source-match');
const existing = await findExistingMirror({
repository,
config,
candidateOwner: repository.mirroredLocation.split('/')[0] || '',
candidateName: repository.name,
});
if (existing) {
await syncGiteaRepo({ config, repository });
console.log(`[Scheduler] Re-synced failed repository with live mirror: ${repo.fullName}`);
return;
}
}
await mirrorGithubRepoToGitea({ octokit, repository, config });
console.log(`[Scheduler] Auto-mirrored repository: ${repo.fullName}`);
} catch (error) {
@@ -431,13 +452,16 @@ async function checkAutoStartConfiguration(): Promise<boolean> {
.where(eq(configs.isActive, true));
for (const config of activeConfigs) {
// Check if scheduling is enabled via environment
// Check if scheduling is enabled.
// Note: env-config-loader already sets scheduleConfig.enabled=true when
// GITEA_MIRROR_INTERVAL is set at startup, so the enabled flag is the
// single authoritative gate here. Checking hasMirrorInterval directly
// would allow a configured interval to trigger auto-start even after the
// user explicitly disabled scheduling via the UI.
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// If either SCHEDULE_ENABLED=true or GITEA_MIRROR_INTERVAL is set, we should auto-start
if (scheduleEnabled || hasMirrorInterval) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled}, hasMirrorInterval=${hasMirrorInterval})`);
if (scheduleEnabled) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled})`);
return true;
}
}
@@ -472,10 +496,12 @@ async function performInitialAutoStart(): Promise<void> {
}
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// Only process configs that have scheduling or mirror interval configured
if (!scheduleEnabled && !hasMirrorInterval) {
// Only process configs where scheduling is explicitly enabled.
// env-config-loader already sets enabled=true when GITEA_MIRROR_INTERVAL
// is present, so this single check covers both the UI toggle and the
// env-var boot path without letting a bare interval override a disabled toggle.
if (!scheduleEnabled) {
continue;
}
-303
View File
@@ -1,303 +0,0 @@
/**
* Enhanced handler for starred repositories with improved error handling
*/
import type { Config, Repository } from "./db/schema";
import { Octokit } from "@octokit/rest";
import { processWithRetry } from "./utils/concurrency";
import {
getOrCreateGiteaOrgEnhanced,
getGiteaRepoInfo,
handleExistingNonMirrorRepo,
createOrganizationsSequentially
} from "./gitea-enhanced";
import { mirrorGithubRepoToGitea } from "./gitea";
import { getMirrorStrategyConfig } from "./utils/mirror-strategies";
import { createMirrorJob } from "./helpers";
/**
* Process starred repositories with enhanced error handling
*/
export async function processStarredRepositories({
config,
repositories,
octokit,
}: {
config: Config;
repositories: Repository[];
octokit: Octokit;
}): Promise<void> {
if (!config.userId) {
throw new Error("User ID is required");
}
const strategyConfig = getMirrorStrategyConfig();
console.log(`Processing ${repositories.length} starred repositories`);
console.log(`Using strategy config:`, strategyConfig);
// Step 1: Pre-create organizations to avoid race conditions
if (strategyConfig.sequentialOrgCreation) {
await preCreateOrganizations({ config, repositories });
}
// Step 2: Process repositories with enhanced error handling
await processWithRetry(
repositories,
async (repository) => {
try {
await processStarredRepository({
config,
repository,
octokit,
strategyConfig,
});
return repository;
} catch (error) {
console.error(`Failed to process starred repository ${repository.name}:`, error);
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 2,
retryDelay: 2000,
onProgress: (completed, total, result) => {
const percentComplete = Math.round((completed / total) * 100);
if (result) {
console.log(
`Processed starred repository "${result.name}" (${completed}/${total}, ${percentComplete}%)`
);
}
},
onRetry: (repo, error, attempt) => {
console.log(
`Retrying starred repository ${repo.name} (attempt ${attempt}): ${error.message}`
);
},
}
);
}
/**
* Pre-create all required organizations sequentially
*/
async function preCreateOrganizations({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
// Get unique organization names
const orgNames = new Set<string>();
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
if (starredReposMode === "preserve-owner") {
for (const repo of repositories) {
orgNames.add(repo.organization || repo.owner);
}
} else if (config.githubConfig?.starredReposOrg) {
orgNames.add(config.githubConfig.starredReposOrg);
} else {
orgNames.add("starred");
}
// Add any other organizations based on mirror strategy
for (const repo of repositories) {
if (repo.destinationOrg) {
orgNames.add(repo.destinationOrg);
}
}
console.log(`Pre-creating ${orgNames.size} organizations sequentially`);
// Create organizations sequentially
await createOrganizationsSequentially({
config,
orgNames: Array.from(orgNames),
});
}
/**
* Process a single starred repository with enhanced error handling
*/
async function processStarredRepository({
config,
repository,
octokit,
strategyConfig,
}: {
config: Config;
repository: Repository;
octokit: Octokit;
strategyConfig: ReturnType<typeof getMirrorStrategyConfig>;
}): Promise<void> {
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
// Check if repository exists in Gitea
const existingRepo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (existingRepo) {
if (existingRepo.mirror) {
console.log(`Starred repository ${repository.name} already exists as a mirror`);
// Update database status
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
} else {
// Repository exists but is not a mirror
console.warn(`Starred repository ${repository.name} exists but is not a mirror`);
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo: existingRepo,
strategy: strategyConfig.nonMirrorStrategy,
});
// If we deleted it, continue to create the mirror
if (strategyConfig.nonMirrorStrategy !== "delete") {
return; // Skip if we're not deleting
}
}
}
// Create the mirror
try {
await mirrorGithubRepoToGitea({
octokit,
repository,
config,
});
} catch (error) {
// Enhanced error handling for specific scenarios
if (error instanceof Error) {
const errorMessage = error.message.toLowerCase();
if (errorMessage.includes("already exists")) {
// Handle race condition where repo was created by another process
console.log(`Repository ${repository.name} was created by another process`);
// Check if it's a mirror now
const recheck = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (recheck && recheck.mirror) {
// It's now a mirror, update database
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
}
}
}
throw error;
}
}
/**
* Sync all starred repositories
*/
export async function syncStarredRepositories({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
const strategyConfig = getMirrorStrategyConfig();
console.log(`Syncing ${repositories.length} starred repositories`);
await processWithRetry(
repositories,
async (repository) => {
try {
// Import syncGiteaRepo
const { syncGiteaRepo } = await import("./gitea");
await syncGiteaRepo({
config,
repository,
});
return repository;
} catch (error) {
if (error instanceof Error && error.message.includes("not a mirror")) {
console.warn(`Repository ${repository.name} is not a mirror, handling...`);
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
const repoInfo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (repoInfo) {
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo,
strategy: strategyConfig.nonMirrorStrategy,
});
}
}
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 1,
retryDelay: 1000,
onProgress: (completed, total) => {
const percentComplete = Math.round((completed / total) * 100);
console.log(`Sync progress: ${completed}/${total} (${percentComplete}%)`);
},
}
);
}
+67
View File
@@ -124,3 +124,70 @@ test("githubConfigSchema parses includeCollaboratorRepos with true default", ()
});
expect(parsed.includeCollaboratorRepos).toBe(true);
});
test("skipPersonalRepos defaults to false in githubConfigSchema", () => {
const parsed = githubConfigSchema.parse({
owner: "octo",
type: "personal",
token: "",
});
expect(parsed.skipPersonalRepos).toBe(false);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when true", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: true };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(true);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(true);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when false", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: false };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(false);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(false);
});
test("DB row missing skipPersonalRepos defaults to false on read", () => {
const ui = mapDbToUiConfig({ githubConfig: { owner: "octo", token: "" } });
expect(ui.advancedOptions.skipPersonalRepos).toBe(false);
});
// Regression for #326: the Name Collision Strategy dropdown didn't persist
// because starredDuplicateStrategy was missing from both mapper directions.
test("starredDuplicateStrategy round-trips UI -> DB -> UI when set to prefix", () => {
const ui = buildMinimalUiConfigs();
ui.githubConfig.starredDuplicateStrategy = "prefix";
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("prefix");
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.githubConfig.starredDuplicateStrategy).toBe("prefix");
});
test("starredDuplicateStrategy round-trips UI -> DB -> UI when set to suffix", () => {
const ui = buildMinimalUiConfigs();
ui.githubConfig.starredDuplicateStrategy = "suffix";
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("suffix");
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
test("starredDuplicateStrategy defaults to suffix on save when unset", () => {
const ui = buildMinimalUiConfigs();
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
test("DB row missing starredDuplicateStrategy defaults to suffix on read", () => {
const ui = mapDbToUiConfig({ githubConfig: { owner: "octo", token: "" } });
expect(ui.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
+25 -2
View File
@@ -31,6 +31,24 @@ function normalizeStarredLists(lists: string[] | undefined): string[] {
return [...deduped];
}
// Trim, drop blanks, and de-duplicate case-insensitively while preserving the
// first-seen casing of each organization name.
function normalizeOrgList(orgs: string[] | undefined): string[] {
if (!Array.isArray(orgs)) return [];
const seen = new Set<string>();
const result: string[] = [];
for (const org of orgs) {
if (typeof org !== "string") continue;
const trimmed = org.trim();
if (!trimmed) continue;
const key = trimmed.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
result.push(trimmed);
}
return result;
}
/**
* Maps UI config structure to database schema structure
*/
@@ -56,13 +74,14 @@ export function mapUiToDbConfig(
includeArchived: false, // Not in UI yet, default to false
includePublic: true, // Not in UI yet, default to true
// Organization related fields
includeOrganizations: [], // Not in UI yet
// Organization related fields — opt-in allowlist (empty = all org repos)
includeOrganizations: normalizeOrgList(githubConfig.includeOrganizations),
// Starred repos organization
starredReposOrg: giteaConfig.starredReposOrg,
starredReposMode: giteaConfig.starredReposMode || "dedicated-org",
starredLists: normalizeStarredLists(githubConfig.starredLists),
starredDuplicateStrategy: githubConfig.starredDuplicateStrategy ?? "suffix",
// Mirror strategy
mirrorStrategy: giteaConfig.mirrorStrategy || "preserve",
@@ -71,6 +90,7 @@ export function mapUiToDbConfig(
// Advanced options
starredCodeOnly: advancedOptions.starredCodeOnly,
autoMirrorStarred: advancedOptions.autoMirrorStarred ?? false,
skipPersonalRepos: advancedOptions.skipPersonalRepos ?? false,
};
// Map Gitea config to match database schema
@@ -144,8 +164,10 @@ export function mapDbToUiConfig(dbConfig: any): {
token: dbConfig.githubConfig?.token || "",
privateRepositories: dbConfig.githubConfig?.includePrivate || false, // Map includePrivate to privateRepositories
includeCollaboratorRepos: dbConfig.githubConfig?.includeCollaboratorRepos ?? true,
includeOrganizations: normalizeOrgList(dbConfig.githubConfig?.includeOrganizations),
mirrorStarred: dbConfig.githubConfig?.includeStarred || false, // Map includeStarred to mirrorStarred
starredLists: normalizeStarredLists(dbConfig.githubConfig?.starredLists),
starredDuplicateStrategy: dbConfig.githubConfig?.starredDuplicateStrategy ?? "suffix",
};
// Map from database Gitea config to UI fields
@@ -194,6 +216,7 @@ export function mapDbToUiConfig(dbConfig: any): {
// Support both old (skipStarredIssues) and new (starredCodeOnly) field names for backward compatibility
starredCodeOnly: dbConfig.githubConfig?.starredCodeOnly ?? (dbConfig.githubConfig as any)?.skipStarredIssues ?? false,
autoMirrorStarred: dbConfig.githubConfig?.autoMirrorStarred ?? false,
skipPersonalRepos: dbConfig.githubConfig?.skipPersonalRepos ?? false,
};
return {
+420
View File
@@ -0,0 +1,420 @@
import { describe, test, expect } from "bun:test";
import {
normalizeCloneUrl,
cloneUrlsMatch,
isMirrorOfSource,
classifyCandidateName,
findExistingMirror,
} from "./mirror-source-match";
import type { Repository } from "@/lib/db/schema";
import type { Config } from "@/types/config";
// Minimal Repository factory for tests. Only the fields read by the helper
// matter (cloneUrl, mirroredLocation, fullName, name).
function makeRepo(overrides: Partial<Repository> = {}): Repository {
return {
id: "repo-1",
userId: "user-1",
configId: "config-1",
name: "Update",
fullName: "NostalgiaForInfinity/Update",
url: "https://github.com/NostalgiaForInfinity/Update",
cloneUrl: "https://github.com/NostalgiaForInfinity/Update.git",
owner: "NostalgiaForInfinity",
organization: undefined,
mirroredLocation: "",
isPrivate: false,
isForked: false,
forkedFrom: undefined,
hasIssues: false,
isStarred: true,
isArchived: false,
size: 0,
hasLFS: false,
hasSubmodules: false,
language: undefined,
description: undefined,
defaultBranch: "main",
visibility: "public",
status: "imported",
lastMirrored: undefined,
errorMessage: undefined,
createdAt: new Date(),
updatedAt: new Date(),
...overrides,
} as unknown as Repository;
}
const config: Partial<Config> = {
userId: "user-1",
giteaConfig: { url: "https://gitea.example.com", token: "t" } as any,
};
describe("normalizeCloneUrl", () => {
test("strips trailing .git", () => {
expect(normalizeCloneUrl("https://github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips embedded credentials", () => {
expect(normalizeCloneUrl("https://x-access-token:ghp_secret@github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips trailing slash", () => {
expect(normalizeCloneUrl("https://github.com/a/b/")).toBe(
"https://github.com/a/b"
);
});
test("lowercases host (and value)", () => {
expect(normalizeCloneUrl("https://GitHub.com/a/b")).toBe(
"https://github.com/a/b"
);
});
test("returns empty string for blank/invalid input", () => {
expect(normalizeCloneUrl("")).toBe("");
expect(normalizeCloneUrl(null)).toBe("");
expect(normalizeCloneUrl(undefined)).toBe("");
});
test("handles scp-style git URLs via fallback", () => {
expect(normalizeCloneUrl("git@github.com:a/b.git")).toBe("git@github.com:a/b");
});
});
describe("cloneUrlsMatch", () => {
test("https vs token-embedded URL match", () => {
expect(
cloneUrlsMatch(
"https://github.com/a/b.git",
"https://x-access-token:tok@github.com/a/b.git"
)
).toBe(true);
});
test(".git suffix and trailing slash differences match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/a/b.git/")
).toBe(true);
});
test("host case-insensitive match", () => {
expect(
cloneUrlsMatch("https://GITHUB.com/a/b", "https://github.com/a/b")
).toBe(true);
});
test("different repos do not match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/c/d")
).toBe(false);
});
test("empty/unknown URL never matches", () => {
expect(cloneUrlsMatch("", "https://github.com/a/b")).toBe(false);
expect(cloneUrlsMatch("https://github.com/a/b", undefined)).toBe(false);
});
});
describe("isMirrorOfSource", () => {
test("true when mirror with matching original_url", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b.git"
)
).toBe(true);
});
test("false when not a mirror", () => {
expect(
isMirrorOfSource(
{ mirror: false, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url is for a different source (phantom fork)", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/other/repo" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url missing (cannot confirm)", () => {
expect(
isMirrorOfSource({ mirror: true } as any, "https://github.com/a/b")
).toBe(false);
});
test("false for null repoInfo", () => {
expect(isMirrorOfSource(null, "https://github.com/a/b")).toBe(false);
});
});
describe("findExistingMirror", () => {
test("reuses existing same-source mirror at base candidate name (#315)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
expect(match!.repoName).toBe("Update");
});
test("reuses via mirroredLocation even when base name differs (strategy change, #309)", async () => {
// Strategy changed; current candidate name would be "Update" under "starred",
// but the historical mirror lives at "myorg/Update-NostalgiaForInfinity".
const repo = makeRepo({
mirroredLocation: "myorg/Update-NostalgiaForInfinity",
});
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "myorg" && repoName === "Update-NostalgiaForInfinity") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("myorg");
expect(match!.repoName).toBe("Update-NostalgiaForInfinity");
});
test("returns null on genuine different-source collision (regression guard #95/#236)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
// Same name, but it mirrors a DIFFERENT source.
return {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("returns null for phantom fork (non-mirror at the name)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return { mirror: false, original_url: "" } as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("falls back to fresh creation when mirroredLocation is stale (Gitea repo deleted)", async () => {
const repo = makeRepo({ mirroredLocation: "starred/Update" });
// Both the recorded location and the base candidate are gone.
const getRepoInfo = async () => null;
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("matches mirror even when original_url is token-embedded / .git-suffixed", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url:
"https://x-access-token:tok@github.com/NostalgiaForInfinity/Update.git",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
});
test("skips a candidate whose lookup throws and still resolves a later candidate", async () => {
const repo = makeRepo({ mirroredLocation: "myorg/Update" });
const getRepoInfo = async ({ owner }: any) => {
if (owner === "myorg") {
throw new Error("network blip");
}
if (owner === "starred") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
});
});
describe("classifyCandidateName — suffix vs reuse decision (#315/#309)", () => {
const SOURCE = "https://github.com/NostalgiaForInfinity/Update.git";
test("free name → available", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: false,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("available");
});
test("name occupied by OUR same-source mirror → reusable (no suffix, #315)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("reusable");
});
test("name occupied by a DIFFERENT source → taken (suffix, regression #95/#236)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("name occupied by a NON-mirror → taken (phantom-fork guard, #309)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: { mirror: false, original_url: "" } as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("our same-source mirror but DB-claimed by ANOTHER repo → taken (per-user separation)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: true,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("free in Gitea but DB-claimed by another concurrent op → taken", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: true,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("existing mirror but unknown source (no sourceCloneUrl) → taken", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: undefined,
})
).toBe("taken");
});
});
+213
View File
@@ -0,0 +1,213 @@
import type { Config } from "@/types/config";
import type { Repository } from "@/lib/db/schema";
import type { GiteaRepoInfo } from "@/lib/gitea-enhanced";
/**
* Source-identity matching for mirror reuse.
*
* Starred (and other) repos were duplicating on every re-mirror because the
* existence check only asked "does a repo with this name exist?" never
* "is the existing repo a mirror of THIS same GitHub source?". This module
* answers the second question so callers can reuse an existing same-source
* mirror instead of generating a suffixed duplicate. See issues #315 / #309.
*/
/**
* Normalize a git clone URL for source-identity comparison.
* Strips embedded credentials, a trailing ".git", a trailing slash, and
* lowercases the host (hosts are case-insensitive; paths are not). Returns an
* empty string for blank/invalid input so callers can treat it as "unknown".
*/
export function normalizeCloneUrl(rawUrl?: string | null): string {
if (typeof rawUrl !== "string") return "";
let url = rawUrl.trim();
if (!url) return "";
try {
const parsed = new URL(url);
// Drop any embedded credentials (e.g. https://user:token@host/...).
parsed.username = "";
parsed.password = "";
const host = parsed.host.toLowerCase();
// Strip trailing slash(es) first so a ".git/" suffix still normalizes.
const path = parsed.pathname.replace(/\/+$/, "").replace(/\.git$/i, "");
return `${parsed.protocol}//${host}${path}`.toLowerCase();
} catch {
// Fall back to best-effort string normalization for non-standard URLs
// (e.g. scp-style git@host:owner/repo). Strip credentials before "@",
// drop ".git"/trailing slash, and lowercase the whole thing.
url = url.replace(/^([a-z]+:\/\/)[^@/]+@/i, "$1");
url = url.replace(/\/+$/, "").replace(/\.git$/i, "");
return url.toLowerCase();
}
}
/**
* Whether two clone URLs point at the same source repository, ignoring
* credentials, ".git" suffix, trailing slash, and host case.
*/
export function cloneUrlsMatch(a?: string | null, b?: string | null): boolean {
const normA = normalizeCloneUrl(a);
const normB = normalizeCloneUrl(b);
if (!normA || !normB) return false;
return normA === normB;
}
/**
* Whether an existing Gitea repo is a mirror of the given GitHub source.
* Uses Gitea's original_url (the recorded migration source) when present;
* if Gitea didn't expose original_url, we cannot positively confirm the
* source and return false (callers then treat the name as a genuine
* collision rather than risk mapping onto an unrelated repo #309).
*/
export function isMirrorOfSource(
repoInfo: GiteaRepoInfo | null,
sourceCloneUrl?: string | null
): boolean {
if (!repoInfo || !repoInfo.mirror) return false;
return cloneUrlsMatch(repoInfo.original_url, sourceCloneUrl);
}
export type CandidateNameClassification = "available" | "reusable" | "taken";
/**
* Classify a candidate mirror name for the suffix-vs-reuse decision in
* generateUniqueRepoName. Pure (all I/O is pre-resolved by the caller):
* - "available": free in Gitea and not DB-claimed by another repo use it
* - "reusable": occupied in Gitea by a mirror of THIS source, not DB-claimed
* by another repo reuse it (no suffix)
* - "taken": occupied by a different source / non-mirror, or DB-claimed by
* another repo must suffix
*
* A DB claim by a DIFFERENT repo always blocks reuse so two users mirroring the
* same source into a shared org stay separated.
*/
export function classifyCandidateName({
existsInGitea,
claimedByOther,
repoInfo,
sourceCloneUrl,
}: {
existsInGitea: boolean;
claimedByOther: boolean;
repoInfo: GiteaRepoInfo | null;
sourceCloneUrl?: string | null;
}): CandidateNameClassification {
if (existsInGitea) {
if (!claimedByOther && isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return "reusable";
}
return "taken";
}
// Not in Gitea, but possibly claimed in the DB by a concurrent operation.
if (claimedByOther) return "taken";
return "available";
}
export interface ExistingMirrorMatch {
owner: string;
repoName: string;
repoInfo: GiteaRepoInfo;
}
/**
* Resolve an existing same-source mirror for a repository, if one exists.
*
* Resolution order (backward compatible):
* 1. The recorded repository.mirroredLocation if it still resolves to a
* live mirror of THIS source, reuse it even when the base candidate name
* differs from the current naming strategy (handles strategy changes #309).
* 2. The provided candidate owner/name if that resolves to a live mirror of
* THIS source, reuse it (handles the self-collision that drove suffixing #315).
*
* Returns null when no live same-source mirror is found (caller should create
* a fresh mirror, generating a unique name if the candidate name is taken by a
* DIFFERENT source).
*/
export async function findExistingMirror({
repository,
config,
candidateOwner,
candidateName,
getRepoInfo,
}: {
repository: Repository;
config: Partial<Config>;
candidateOwner: string;
candidateName: string;
// Injectable for testing; defaults to the real Gitea lookup.
getRepoInfo?: (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => Promise<GiteaRepoInfo | null>;
}): Promise<ExistingMirrorMatch | null> {
const lookup =
getRepoInfo ??
(async (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => {
const { getGiteaRepoInfo } = await import("@/lib/gitea-enhanced");
return getGiteaRepoInfo(args);
});
const sourceCloneUrl = repository.cloneUrl;
// Candidate locations to probe, in priority order. Dedupe so we don't issue
// the same HTTP lookup twice when mirroredLocation equals the candidate.
const candidates: Array<{ owner: string; repoName: string }> = [];
const seen = new Set<string>();
const pushCandidate = (owner?: string | null, repoName?: string | null) => {
const o = (owner || "").trim();
const r = (repoName || "").trim();
if (!o || !r) return;
const key = `${o}/${r}`.toLowerCase();
if (seen.has(key)) return;
seen.add(key);
candidates.push({ owner: o, repoName: r });
};
if (repository.mirroredLocation && repository.mirroredLocation.trim()) {
const slashIndex = repository.mirroredLocation.indexOf("/");
if (slashIndex > 0 && slashIndex < repository.mirroredLocation.length - 1) {
pushCandidate(
repository.mirroredLocation.slice(0, slashIndex),
repository.mirroredLocation.slice(slashIndex + 1)
);
}
}
pushCandidate(candidateOwner, candidateName);
for (const candidate of candidates) {
let repoInfo: GiteaRepoInfo | null;
try {
repoInfo = await lookup({
config,
owner: candidate.owner,
repoName: candidate.repoName,
});
} catch (error) {
// A failed lookup (network/auth) should not be mistaken for "no mirror";
// skip this candidate and let the caller fall back to its normal flow.
console.warn(
`[Mirror] Could not look up ${candidate.owner}/${candidate.repoName} while resolving existing mirror for ${repository.fullName}: ${
error instanceof Error ? error.message : String(error)
}`
);
continue;
}
if (isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return {
owner: candidate.owner,
repoName: candidate.repoName,
repoInfo: repoInfo as GiteaRepoInfo,
};
}
}
return null;
}
+89
View File
@@ -0,0 +1,89 @@
import type { APIRoute } from "astro";
import { db, repositories } from "@/lib/db";
import { and, eq, inArray } from "drizzle-orm";
import { repoStatusEnum } from "@/types/Repository";
import { createMirrorJob } from "@/lib/helpers";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
/**
* POST /api/job/cancel-pending
*
* Sets this user's repositories that are waiting to be mirrored
* (status: "imported" or "failed") to "ignored", preventing the scheduler
* from picking them up. Repos with status "mirroring" or "syncing" are
* left alone because they have in-flight work that cannot be aborted here.
*
* Returns the count of affected repositories and logs one activity entry.
*/
export const POST: APIRoute = async ({ request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
// Statuses that represent queued-but-not-started work.
// "imported" → repo was discovered, never mirrored
// "failed" → last mirror attempt failed; scheduler will retry
const cancelableStatuses = ["imported", "failed"] as const;
// Fetch repos to cancel so we can count them and log meaningful details.
const toCancel = await db
.select({ id: repositories.id })
.from(repositories)
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.status, cancelableStatuses),
),
);
const cancelCount = toCancel.length;
if (cancelCount > 0) {
const ids = toCancel.map((r) => r.id);
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("ignored"),
updatedAt: new Date(),
errorMessage: "Cancelled by user — set to ignored via Stop Pending Mirrors.",
})
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.id, ids),
),
);
}
// Log a single activity summarising the bulk action.
await createMirrorJob({
userId,
message: `Stopped pending mirrors: ${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} set to Ignored`,
details:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} with status "imported" or "failed" have been set to "ignored". ` +
`They can be re-enabled from the Repositories page.`
: `No repositories in a pending state were found for this user.`,
status: cancelCount > 0 ? "ignored" : "skipped",
skipDuplicateEvent: false,
skipNotification: true,
});
return new Response(
JSON.stringify({
success: true,
message:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y has" : "ies have"} been set to Ignored.`
: "No repositories in a pending state were found.",
cancelledCount: cancelCount,
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
} catch (error) {
return createSecureErrorResponse(error, "cancel pending mirrors", 500);
}
};
+1 -51
View File
@@ -1,5 +1,5 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { db, repositories } from "@/lib/db";
import { eq, and } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
@@ -62,53 +62,3 @@ export const PATCH: APIRoute = async (context) => {
}
};
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const repoId = context.params.id;
if (!repoId) {
return new Response(JSON.stringify({ error: "Repository ID is required" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
const [existingRepo] = await db
.select()
.from(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)))
.limit(1);
if (!existingRepo) {
return new Response(
JSON.stringify({ error: "Repository not found" }),
{
status: 404,
headers: { "Content-Type": "application/json" },
}
);
}
await db
.delete(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)));
await db
.delete(mirrorJobs)
.where(and(eq(mirrorJobs.repositoryId, repoId), eq(mirrorJobs.userId, userId)));
return new Response(
JSON.stringify({ success: true }),
{
status: 200,
headers: { "Content-Type": "application/json" },
}
);
} catch (error) {
return createSecureErrorResponse(error, "Delete repository", 500);
}
};
+49
View File
@@ -0,0 +1,49 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { eq, and, inArray } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const body = await context.request.json();
const { ids } = body;
if (!Array.isArray(ids) || ids.length === 0) {
return new Response(JSON.stringify({ error: "ids must be a non-empty array" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
// Verify all repos belong to this user before deleting
const owned = await db
.select({ id: repositories.id })
.from(repositories)
.where(and(inArray(repositories.id, ids), eq(repositories.userId, userId)));
const ownedIds = owned.map((r) => r.id);
if (ownedIds.length === 0) {
return new Response(JSON.stringify({ error: "No matching repositories found" }), {
status: 404,
headers: { "Content-Type": "application/json" },
});
}
await db.transaction(async (tx) => {
await tx.delete(mirrorJobs).where(and(inArray(mirrorJobs.repositoryId, ownedIds), eq(mirrorJobs.userId, userId)));
await tx.delete(repositories).where(and(inArray(repositories.id, ownedIds), eq(repositories.userId, userId)));
});
return new Response(
JSON.stringify({ success: true, deleted: ownedIds.length }),
{ status: 200, headers: { "Content-Type": "application/json" } }
);
} catch (error) {
return createSecureErrorResponse(error, "Bulk delete repositories", 500);
}
};
+12 -3
View File
@@ -49,13 +49,20 @@ export const POST: APIRoute = async ({ request, locals }) => {
const githubUsername = config.githubConfig?.owner || undefined;
const octokit = createGitHubClient(decryptedToken, userId, githubUsername);
// Load ignored orgs from the DB so we can skip them during import
const ignoredOrgRows = await db
.select({ normalizedName: organizations.normalizedName })
.from(organizations)
.where(and(eq(organizations.userId, userId), eq(organizations.status, "ignored")));
const ignoredOrgNames = new Set(ignoredOrgRows.map((o) => o.normalizedName));
// Fetch GitHub data in parallel
const [basicAndForkedRepos, starredRepos, orgResult] = await Promise.all([
getGithubRepositories({ octokit, config }),
config.githubConfig?.includeStarred
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
getGithubOrganizations({ octokit, config }),
getGithubOrganizations({ octokit, config, skipOrgNames: ignoredOrgNames }),
]);
const { organizations: gitOrgs, failedOrgs } = orgResult;
@@ -152,7 +159,9 @@ export const POST: APIRoute = async ({ request, locals }) => {
const existingOrgMap = new Map(existingOrgs.map((o) => [o.normalizedName, o.status]));
insertedRepos = newRepos.filter(
(r) => !existingRepoNames.has(r.normalizedFullName)
(r) =>
!existingRepoNames.has(r.normalizedFullName) &&
(!r.organization || !ignoredOrgNames.has(r.organization.toLowerCase()))
);
insertedOrgs = newOrgs.filter((o) => !existingOrgMap.has(o.normalizedName));
@@ -258,7 +267,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
newRepositories: insertedRepos.length,
newOrganizations: insertedOrgs.length,
skippedDisabledRepositories: allGithubRepos.length - mirrorableGithubRepos.length,
failedOrgs: failedOrgs.map((o) => o.name),
failedOrgs: failedOrgs.filter((o) => !ignoredOrgNames.has(o.name.toLowerCase())).map((o) => o.name),
recoveredOrgs: recoveredOrgCount,
},
});
+2
View File
@@ -62,6 +62,7 @@ export interface GitHubConfig {
token: string;
privateRepositories: boolean;
includeCollaboratorRepos?: boolean;
includeOrganizations?: string[];
mirrorStarred: boolean;
starredLists?: string[];
starredDuplicateStrategy?: DuplicateNameStrategy;
@@ -86,6 +87,7 @@ export interface AdvancedOptions {
skipForks: boolean;
starredCodeOnly: boolean;
autoMirrorStarred?: boolean;
skipPersonalRepos?: boolean;
}
export interface SaveConfigApiRequest {
+11 -11
View File
@@ -9,24 +9,24 @@
"astro": "astro"
},
"dependencies": {
"@astrojs/mdx": "^5.0.0",
"@astrojs/react": "^5.0.0",
"@radix-ui/react-slot": "^1.2.4",
"@astrojs/mdx": "^5.0.6",
"@astrojs/react": "^5.0.7",
"@radix-ui/react-slot": "^1.2.5",
"@splinetool/react-spline": "^4.1.0",
"@splinetool/runtime": "^1.12.69",
"@tailwindcss/vite": "^4.2.1",
"@splinetool/runtime": "^1.12.97",
"@tailwindcss/vite": "^4.3.1",
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.2.14",
"@types/react": "^19.2.17",
"@types/react-dom": "^19.2.3",
"astro": "^6.1.6",
"astro": "^6.4.6",
"canvas-confetti": "^1.9.4",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^0.577.0",
"react": "^19.2.4",
"react-dom": "^19.2.4",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.2.1"
"react": "^19.2.7",
"react-dom": "^19.2.7",
"tailwind-merge": "^3.6.0",
"tailwindcss": "^4.3.1"
},
"devDependencies": {
"tw-animate-css": "^1.4.0"
+741 -745
View File
File diff suppressed because it is too large Load Diff