Compare commits

...

18 Commits

Author SHA1 Message Date
Arunavo Ray 632bbd0d4a chore: bump version to 3.20.2 2026-06-24 17:19:10 +05:30
ARUNAVO RAY 0b65e40784 fix(releases): create releases only for tags present in Gitea; stop sending target (#331) (#333)
Release creation failed on some Gitea/Forgejo instances with
"HTTP 404: The target couldn't be found", so no release (and therefore no
assets) was ever created — re-syncing never recovered.

Root cause: the create payload always sent `target: target_commitish`
(e.g. "main"). When the release's git tag is not yet present in the Gitea
mirror — which happens when Gitea's own git mirror clone lags behind the
metadata sync — Gitea tries to *create* the tag from `target`; if that ref
can't be resolved it returns a generic 404 ("The target couldn't be
found"), and if it can, it would create a brand-new tag at the wrong commit.

Reproduced the reporter's exact stack (Forgejo 15 rootless + read_only +
cap_drop ALL + postgres, plus Gitea 1.20-1.26 and Forgejo 1.21-15): a
healthy repo always succeeds — the 404 only occurs when the tag is absent
at create time.

Fix:
- Before creating a release, verify the git tag already exists in Gitea.
  If it isn't synced yet, skip it (logged) and let a later sync create it
  once the mirror has the tag — never create a tag via `target`.
- Drop the `target` field from both the create and update payloads. For a
  mirror the tag is synced from upstream, so Gitea attaches the release to
  the existing tag; `target` is unnecessary and is what triggers the 404.
- Surface skipped-missing-tag releases in the summary log for diagnosability.

Verified end-to-end against the real mirror function on a Forgejo instance:
a release whose tag exists is created with its assets; a release whose tag
was removed is skipped cleanly (no 404, no bogus tag) and picked up once the
tag is present.
2026-06-24 17:18:46 +05:30
Arunavo Ray 4a8b4f6ff3 chore: bump version to 3.20.1 2026-06-23 23:07:16 +05:30
ARUNAVO RAY 1d9dfdeb70 fix(releases): mirror assets idempotently so missing assets self-heal (#331) (#332)
Release assets were uploaded only on the create path of
mirrorGitHubReleasesToGitea(). When a Gitea release already existed, the
update path PATCHed the changelog/title and `continue`d without ever
touching assets. So any release whose assets were not fully uploaded on
that single create-path run — first sync interrupted, a transient
download/upload failure, large multi-MB assets, etc. — stayed permanently
asset-less, and re-syncing always hit the update path and could never
recover it. Asset failures were also swallowed to console.error, so the
job still reported success (the "no errors in the logs" in #331).

Reproduced on a real Forgejo pull-mirror with shauninman/MinUI: a GitHub
release with two ~35-40MB binaries became a Gitea release with 0 assets
(just Forgejo's auto-generated source archive), and re-syncing left it at
0 while logging "Updating existing release".

Fix:
- Add reconcileReleaseAssets(), an idempotent reconciler run on BOTH the
  create and update paths. It compares Gitea's existing attachments to
  GitHub's by name+size: skips matches, uploads missing ones, replaces
  size-mismatched copies. Existing broken releases self-heal on next sync.
- Extract the pure decision into classifyAssetsForReconciliation() for
  unit testing (the absence of asset tests is why this slipped past #310).
- Surface asset upload counts in the summary and emit a visible warning
  when any fail, instead of silently swallowing them.
- Add 5 unit tests covering the broken state, partial backfill,
  idempotency, size-mismatch replacement, and the no-assets case.

Verified end-to-end against the real function on a Forgejo pull-mirror:
0 -> 2 assets backfilled, already-present assets skipped (no re-download),
second run uploads 0.
2026-06-23 23:06:43 +05:30
Arunavo Ray 079df29f44 chore: bump version to 3.20.0 2026-06-19 08:44:03 +05:30
ARUNAVO RAY dff3cafb5e fix(config): persist Name Collision Strategy (starredDuplicateStrategy) (#326) (#328)
The "Name collision strategy" dropdown (starredDuplicateStrategy) never
persisted: the field was absent from both directions of the UI<->DB config
mapper. On save, mapUiToDbConfig dropped it before the DB write; on load,
mapDbToUiConfig never read it, so the UI reset to the "suffix" (repo-owner)
default. Mirror logic in gitea.ts then read undefined and also defaulted to
suffix — so repos really were created with that pattern regardless of the
user's choice. It has been broken since the field was introduced.

- Map starredDuplicateStrategy in mapUiToDbConfig and mapDbToUiConfig
- Add STARRED_DUPLICATE_STRATEGY env var for parity (reporter could not
  work around it via compose because no env var existed) + docs
- Round-trip tests covering save, load, and the missing-field default
2026-06-19 08:43:27 +05:30
ARUNAVO RAY 6ca7c0eec0 feat(github): add organization allowlist to mirror only selected orgs (#327)
Repository discovery requested the `organization_member` affiliation
unconditionally, so repos from every org a user belongs to were imported —
even orgs they never explicitly added. `skipPersonalRepos` only dropped
user-owned repos and left org repos unfiltered, which surprised users who
expected "only mirror org repos" to mean "only the orgs I chose" (reported
on #304).

Wire up the previously-dormant `includeOrganizations` config field as an
opt-in allowlist: when non-empty, only repos owned by the listed
organizations are imported. Empty = all org repos (backward-compatible).
Owned and collaborator repos are never restricted, so it composes cleanly
with `skipPersonalRepos`.

- Filter org repos by the allowlist in getGithubRepositories
- Add includeAllOrgsOverride so the cleanup service bypasses the allowlist
  and never false-orphans a previously-mirrored repo from an org the user
  later removes from the list
- UI control under Filtering & Behavior; INCLUDE_ORGANIZATIONS env var
- Case-insensitive dedup/trim in the UI<->DB mapper round-trip
- 7 unit tests covering the filter, composition, and the cleanup override
2026-06-19 08:42:17 +05:30
ARUNAVO RAY 6ebf1916e8 chore(deps): update dependencies across app and website (#325)
* chore(deps): update app dependencies to latest in-range versions

* chore(deps): update website (www) dependencies to latest in-range versions
2026-06-14 12:25:45 +05:30
Arunavo Ray 91de0d1030 chore: bump version to 3.19.1 2026-06-14 12:07:48 +05:30
Brendan Davidson 85bd1f4042 Repository table bulk actions (#322)
* Handle indexing when shift + clicking in the repository table

* Move the buttons when selecting rows

* Add in a bulk delete func in the repositories table

* Add bulk delete handler

* Make the single action use the bulk delete

* Delete the single repository id handler
2026-06-14 10:14:51 +05:30
Brendan Davidson 4a28015685 Skip the user defined orgs to ignore (#323) 2026-06-14 10:14:48 +05:30
Arunavo Ray da23941369 chore: bump version to 3.19.0 2026-06-13 09:14:33 +05:30
Brendan Davidson 906ce57e8c Handle indexing when shift + clicking in the repository table (#316) 2026-06-13 09:14:02 +05:30
Arunavo Ray 1b84c75a97 chore: bump version to 3.18.0 2026-06-13 08:17:20 +05:30
ARUNAVO RAY 0b6b6b76bf feat(github): add skipPersonalRepos toggle to mirror only org repos (#304) (#320)
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
2026-06-13 08:00:50 +05:30
ARUNAVO RAY 7610a614da fix: scheduler auto-start gate, backup clone URL, cancel-pending action, actionable 405 (#319)
* fix(scheduler): make enabled flag authoritative for auto-start

checkAutoStartConfiguration() and performInitialAutoStart() previously
used `scheduleEnabled || hasMirrorInterval`, allowing a configured
GITEA_MIRROR_INTERVAL to trigger boot-time auto-start even after the
user disabled scheduling via the UI toggle.

env-config-loader already writes scheduleConfig.enabled=true when
GITEA_MIRROR_INTERVAL is set at container startup, so the interval is
a timing detail, not an enable signal. The documented env-var contract
is preserved: GITEA_MIRROR_INTERVAL at boot → env-config-loader sets
enabled=true → auto-start fires. But a later UI disable now sticks.

Add a focused unit test for the gate logic.

* fix(backup): always derive clone URL from user-configured Gitea URL

The pre-sync backup preferred repoInfo.clone_url, which reflects
Gitea's ROOT_URL setting. In Tailscale MagicDNS deployments (and any
setup where ROOT_URL is an external address), this URL is unreachable
from the app itself, causing bundle backup to fail.

Always build the clone URL as:
  ${config.giteaConfig.url.trimEnd('/')}/${owner}/${repo}.git

This matches the URL the app already uses for all other Gitea API
calls and is guaranteed reachable.

* feat(jobs): cancel-pending endpoint + fix misleading Delete All copy

Add POST /api/job/cancel-pending that sets the current user's
repositories with status "imported" or "failed" to "ignored",
preventing the scheduler from re-queuing them. In-flight "mirroring"
rows are left alone. Returns the count and logs one activity entry.

Fix the "Delete All Activities" dialog to clearly state it only clears
the history log and does not stop pending work. Rename button/title to
"Clear History" so intent is unambiguous.

Add a "Stop Pending Mirrors" button (StopCircle icon, amber) in both
mobile and desktop activity log toolbars, with a confirmation dialog
explaining repos are set to Ignored and can be re-enabled from the
Repositories page.

* fix(sync): actionable 405 error for non-pull-mirror repos

Gitea returns HTTP 405 with an empty body when the target repository is
no longer a pull mirror — e.g. the mirror was auto-disabled by Gitea or
the repository lost its mirror state after a manual edit.

Previously this fell through to the generic error handler which stored
the raw HttpError message (often empty) giving the user no guidance.

Now a 405 response is caught alongside the existing 400 handler and
sets the repository to "failed" with an actionable error message:

  "Gitea reports this repository is not a pull mirror (HTTP 405).
  In Gitea check Settings → Mirror Settings; if the mirror section is
  missing, delete the repository in Gitea and re-mirror it from
  gitea-mirror."

The same message is written to the activity log for visibility in the
dashboard.
2026-06-13 08:00:47 +05:30
ARUNAVO RAY c28dcc209f fix(releases): stop delete/recreate cycle on permanent order mismatch (#310) (#318)
Root cause (Theory A): the `needsRecreation` check compared GitHub
published_at-based expected indices against Gitea's API order. Gitea mirror
repos sort releases by tag-commit date, which can permanently disagree with
published_at order (e.g. unaconfig_dart v0.1.0 published after v0.1.1 but
tagged before). This made `currentExpectedIdx < nextExpectedIdx` evaluate
true on every sync, triggering delete-all-and-recreate forever — spamming
Gitea's activity feed with "released X" events (#310).

Fix: replace the destructive order-check machinery with set-based
reconciliation via `classifyReleasesForReconciliation`. Releases are
created when missing in Gitea and skipped (or PATCH-updated if content
drifted) when already present. No deletions are ever triggered by ordering.
Retain the existing release-limit trimming (retention cleanup) unchanged.

Also removes the 1-second per-release delay that was only needed for the
creation-order dance, significantly speeding up initial mirrors.

Adds unit tests covering: normal ordered repos, the unaconfig_dart inversion
fixture, missing→create, present→skip, and edge cases.
2026-06-13 08:00:44 +05:30
ARUNAVO RAY 40ee3cbc44 fix(mirror): reuse existing same-source mirrors instead of creating suffixed duplicates (#315) (#317)
Starred (and other) repos duplicated on every re-mirror (starred/Repo,
Repo-owner, Repo-owner-1, ...) because the existence check only asked
"does a repo with this name exist?" and never "is the existing repo a
mirror of THIS same source?". The repo's own prior mirror counted as a
collision, so generateUniqueRepoName bumped to the next suffix each run,
repointing mirroredLocation at the newest copy. Under a single re-call,
3 concurrent/retried jobs each computed a DIFFERENT suffixed name, so the
location-based in-flight guard never matched and the race produced extra
copies.

Fix (source-identity aware):
- New shared helper src/lib/utils/mirror-source-match.ts:
  - normalizeCloneUrl / cloneUrlsMatch: credential-, .git-, slash- and
    host-case-insensitive clone URL comparison.
  - isMirrorOfSource: a Gitea repo is "ours" only if it is a mirror AND
    its original_url matches this repo's source.
  - findExistingMirror: resolves an existing same-source mirror via the
    recorded mirroredLocation first (survives strategy changes — #309),
    then the base candidate name.
  - classifyCandidateName: pure available/reusable/taken decision.
- gitea-enhanced: export GiteaRepoInfo and add original_url (Gitea's
  recorded migration source) for source matching.
- Both create paths (mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg):
  run findExistingMirror BEFORE name generation; on a hit, reuse that
  location and route into the existing "already mirrored" handling rather
  than calling generateUniqueRepoName. Names now converge under
  concurrency so the in-flight guard becomes effective.
- generateUniqueRepoName is now source-aware: an occupied name held by a
  mirror of the SAME source is reused (no suffix); suffixing only happens
  on a genuine different-source collision, preserving #95/#236 behavior.
  The per-user DB claim check is retained so two users mirroring the same
  source into a shared org stay separated.
- Phantom-fork guard (#309): the existingRepoInfo.mirror branches now
  verify same-source before marking "mirrored"; on mismatch they fall
  through to unique-name generation and create a separate mirror.
- Scheduler: a `failed` repo whose mirroredLocation still resolves to a
  live same-source mirror is routed to syncGiteaRepo instead of re-create,
  breaking the failed-metadata re-create loop cheaply.
- Remove dead src/lib/starred-repos-handler.ts (zero importers across all
  git history); its correct base-name/.mirror reuse logic now lives in the
  shared helper.

Tests: src/lib/utils/mirror-source-match.test.ts (30 cases) covers URL
normalization, reuse at base name, reuse via mirroredLocation across a
strategy change, genuine different-source collision (suffix), phantom
fork, stale mirroredLocation fallback, per-user DB-claim separation, and
the suffix-vs-reuse classification. Full suite: 319 pass, 0 fail.
2026-06-13 08:00:41 +05:30
29 changed files with 3524 additions and 1874 deletions
+318 -316
View File
File diff suppressed because it is too large Load Diff
+3 -1
View File
@@ -107,6 +107,7 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
| `MIRROR_STARRED_LISTS` | Optional comma-separated GitHub Star List names to mirror (only used when `MIRROR_STARRED=true`) | - | Comma-separated list names (empty = all starred repos) |
| `STARRED_REPOS_ORG` | Organization name for starred repos | `starred` | Any string |
| `STARRED_REPOS_MODE` | How starred repos are mirrored | `dedicated-org` | `dedicated-org`, `preserve-owner` |
| `STARRED_DUPLICATE_STRATEGY` | Name collision strategy when two starred repos share a name from different owners (`suffix` = `repo-owner`, `prefix` = `owner-repo`) | `suffix` | `suffix`, `prefix`, `owner-org` |
### Organization Settings
@@ -114,7 +115,8 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
|----------|-------------|---------|---------|
| `MIRROR_ORGANIZATIONS` | Mirror organization repositories | `false` | `true`, `false` |
| `PRESERVE_ORG_STRUCTURE` | Preserve GitHub organization structure in Gitea | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal) | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal); sets `skipPersonalRepos: true` in GitHub config | `false` | `true`, `false` |
| `INCLUDE_ORGANIZATIONS` | Opt-in allowlist: only mirror repos from these organizations (empty = all orgs you belong to). Sets `includeOrganizations` in GitHub config | - | Comma-separated org names |
| `MIRROR_STRATEGY` | Repository organization strategy | `preserve` | `preserve`, `single-org`, `flat-user`, `mixed` |
### Advanced Settings
+43 -43
View File
@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.17.1",
"version": "3.20.2",
"engines": {
"bun": ">=1.2.9"
},
@@ -54,39 +54,39 @@
"picomatch": "^4.0.4"
},
"dependencies": {
"@astrojs/check": "^0.9.7",
"@astrojs/check": "^0.9.9",
"@astrojs/mdx": "5.0.0",
"@astrojs/node": "10.0.1",
"@astrojs/react": "^5.0.0",
"@astrojs/node": "10.1.4",
"@astrojs/react": "^5.0.7",
"@better-auth/oauth-provider": "1.6.11",
"@better-auth/sso": "1.6.11",
"@octokit/plugin-throttling": "^11.0.3",
"@octokit/rest": "^22.0.1",
"@radix-ui/react-accordion": "^1.2.12",
"@radix-ui/react-avatar": "^1.1.11",
"@radix-ui/react-checkbox": "^1.3.3",
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-dialog": "^1.1.15",
"@radix-ui/react-dropdown-menu": "^2.1.16",
"@radix-ui/react-hover-card": "^1.1.15",
"@radix-ui/react-label": "^2.1.8",
"@radix-ui/react-popover": "^1.1.15",
"@radix-ui/react-progress": "^1.1.8",
"@radix-ui/react-radio-group": "^1.3.8",
"@radix-ui/react-scroll-area": "^1.2.10",
"@radix-ui/react-select": "^2.2.6",
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
"@radix-ui/react-switch": "^1.2.6",
"@radix-ui/react-tabs": "^1.1.13",
"@radix-ui/react-tooltip": "^1.2.8",
"@tailwindcss/vite": "^4.2.1",
"@radix-ui/react-accordion": "^1.2.13",
"@radix-ui/react-avatar": "^1.1.12",
"@radix-ui/react-checkbox": "^1.3.4",
"@radix-ui/react-collapsible": "^1.1.13",
"@radix-ui/react-dialog": "^1.1.16",
"@radix-ui/react-dropdown-menu": "^2.1.17",
"@radix-ui/react-hover-card": "^1.1.16",
"@radix-ui/react-label": "^2.1.9",
"@radix-ui/react-popover": "^1.1.16",
"@radix-ui/react-progress": "^1.1.9",
"@radix-ui/react-radio-group": "^1.4.0",
"@radix-ui/react-scroll-area": "^1.2.11",
"@radix-ui/react-select": "^2.3.0",
"@radix-ui/react-separator": "^1.1.9",
"@radix-ui/react-slot": "^1.2.5",
"@radix-ui/react-switch": "^1.3.0",
"@radix-ui/react-tabs": "^1.1.14",
"@radix-ui/react-tooltip": "^1.2.9",
"@tailwindcss/vite": "^4.3.1",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.19",
"@tanstack/react-virtual": "^3.14.2",
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.2.14",
"@types/react": "^19.2.17",
"@types/react-dom": "^19.2.3",
"astro": "^6.0.4",
"astro": "^6.4.6",
"bcryptjs": "^3.0.3",
"better-auth": "1.6.11",
"buffer": "^6.0.3",
@@ -94,39 +94,39 @@
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"dotenv": "^17.3.1",
"dotenv": "^17.4.2",
"drizzle-orm": "^0.45.2",
"fuse.js": "^7.1.0",
"fuse.js": "^7.4.2",
"jsonwebtoken": "^9.0.3",
"lucide-react": "^0.577.0",
"nanoid": "^5.1.6",
"nanoid": "^5.1.11",
"next-themes": "^0.4.6",
"react": "^19.2.4",
"react-dom": "^19.2.4",
"react-icons": "^5.5.0",
"react": "^19.2.7",
"react-dom": "^19.2.7",
"react-icons": "^5.6.0",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.2.1",
"tailwind-merge": "^3.6.0",
"tailwindcss": "^4.3.1",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.3",
"uuid": "^13.0.0",
"uuid": "^13.0.2",
"vaul": "^1.1.2",
"zod": "^4.3.6"
"zod": "^4.4.3"
},
"devDependencies": {
"@playwright/test": "^1.58.2",
"@playwright/test": "^1.60.0",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^16.3.2",
"@types/bcryptjs": "^3.0.0",
"@types/bun": "^1.3.10",
"@types/bun": "^1.3.14",
"@types/jsonwebtoken": "^9.0.10",
"@types/node": "^25.5.0",
"@types/node": "^25.9.3",
"@types/uuid": "^11.0.0",
"@vitejs/plugin-react": "^6.0.1",
"drizzle-kit": "^0.31.9",
"@vitejs/plugin-react": "^6.0.2",
"drizzle-kit": "^0.31.10",
"jsdom": "^28.1.0",
"tsx": "^4.21.0",
"vitest": "^4.1.0"
"tsx": "^4.22.4",
"vitest": "^4.1.8"
},
"packageManager": "bun@1.3.10"
}
+97 -7
View File
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useState, useRef } from 'react';
import { Button } from '@/components/ui/button';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter } from 'lucide-react';
import { ChevronDown, Download, RefreshCw, Search, Trash2, Filter, StopCircle } from 'lucide-react';
import {
DropdownMenu,
DropdownMenuContent,
@@ -85,6 +85,8 @@ export function ActivityLog() {
const [activities, setActivities] = useState<MirrorJobWithKey[]>([]);
const [isInitialLoading, setIsInitialLoading] = useState(false);
const [showCleanupDialog, setShowCleanupDialog] = useState(false);
const [showCancelPendingDialog, setShowCancelPendingDialog] = useState(false);
const [isCancelPendingLoading, setIsCancelPendingLoading] = useState(false);
// Ref to track if component is mounted to prevent state updates after unmount
const isMountedRef = useRef(true);
@@ -354,6 +356,40 @@ export function ActivityLog() {
setShowCleanupDialog(false);
};
const confirmCancelPending = async () => {
if (!user?.id) return;
try {
setIsCancelPendingLoading(true);
setShowCancelPendingDialog(false);
const response = await fetch(withBase('/api/job/cancel-pending'), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
});
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: 'Unknown error occurred' }));
throw new Error(errorData.error || `HTTP ${response.status}: ${response.statusText}`);
}
const res = await response.json();
if (res.success) {
toast.success(res.message);
// Refresh to show the new activity log entry
await fetchActivities(false);
} else {
showErrorToast(res.error || 'Failed to cancel pending mirrors.', toast);
}
} catch (error) {
console.error('Error cancelling pending mirrors:', error);
showErrorToast(error, toast);
} finally {
setIsCancelPendingLoading(false);
}
};
// Check if any filters are active
const hasActiveFilters = !!(filter.status || filter.type || filter.name);
const activeFilterCount = [filter.status, filter.type, filter.name].filter(Boolean).length;
@@ -552,11 +588,22 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10 shrink-0"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10 shrink-0"
>
<Trash2 className="h-4 w-4" />
@@ -683,12 +730,24 @@ export function ActivityLog() {
<RefreshCw className="h-4 w-4" />
</Button>
{/* cleanup all activities */}
{/* stop pending mirrors */}
<Button
variant="outline"
size="icon"
onClick={() => setShowCancelPendingDialog(true)}
title="Stop pending mirrors"
className="text-amber-600 hover:text-amber-600 h-10 w-10"
disabled={isCancelPendingLoading}
>
<StopCircle className="h-4 w-4" />
</Button>
{/* clear activity history */}
<Button
variant="outline"
size="icon"
onClick={handleCleanupClick}
title="Delete all activities"
title="Clear activity history"
className="text-destructive hover:text-destructive h-10 w-10"
>
<Trash2 className="h-4 w-4" />
@@ -709,9 +768,12 @@ export function ActivityLog() {
<Dialog open={showCleanupDialog} onOpenChange={setShowCleanupDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete All Activities</DialogTitle>
<DialogTitle>Clear Activity History</DialogTitle>
<DialogDescription>
Are you sure you want to delete ALL activities? This action cannot be undone and will remove all mirror jobs and events from the database.
This clears the activity <strong>history log</strong> (mirror job records and events) it does not stop
any pending or in-progress mirrors. Repositories keep their current status and the scheduler
will continue to pick up pending work. To stop pending mirrors, use the
&ldquo;Stop Pending Mirrors&rdquo; button instead.
</DialogDescription>
</DialogHeader>
<DialogFooter>
@@ -723,7 +785,35 @@ export function ActivityLog() {
onClick={confirmCleanup}
disabled={isInitialLoading}
>
{isInitialLoading ? 'Deleting...' : 'Delete All Activities'}
{isInitialLoading ? 'Clearing...' : 'Clear History'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* cancel pending mirrors confirmation dialog */}
<Dialog open={showCancelPendingDialog} onOpenChange={setShowCancelPendingDialog}>
<DialogContent>
<DialogHeader>
<DialogTitle>Stop Pending Mirrors</DialogTitle>
<DialogDescription>
This sets all repositories with status <strong>Imported</strong> or <strong>Failed</strong> to{' '}
<strong>Ignored</strong>, preventing the scheduler from mirroring them automatically.
Repositories that are currently mirroring are not affected.{' '}
You can re-enable individual repositories from the Repositories page.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button variant="outline" onClick={() => setShowCancelPendingDialog(false)}>
Cancel
</Button>
<Button
variant="default"
className="bg-amber-600 hover:bg-amber-700"
onClick={confirmCancelPending}
disabled={isCancelPendingLoading}
>
{isCancelPendingLoading ? 'Stopping...' : 'Stop Pending Mirrors'}
</Button>
</DialogFooter>
</DialogContent>
@@ -77,6 +77,7 @@ export function GitHubMirrorSettings({
const [starListsOpen, setStarListsOpen] = React.useState(false);
const [starListSearch, setStarListSearch] = React.useState("");
const [customStarListName, setCustomStarListName] = React.useState("");
const [customOrgName, setCustomOrgName] = React.useState("");
const [availableStarLists, setAvailableStarLists] = React.useState<string[]>([]);
const [loadingStarLists, setLoadingStarLists] = React.useState(false);
const [loadedStarLists, setLoadedStarLists] = React.useState(false);
@@ -138,6 +139,38 @@ export function GitHubMirrorSettings({
});
}, [githubConfig, normalizeStarListNames, onGitHubConfigChange]);
const includedOrgs = React.useMemo(
() => githubConfig.includeOrganizations ?? [],
[githubConfig.includeOrganizations],
);
const addIncludedOrg = React.useCallback(() => {
const trimmed = customOrgName.trim();
if (!trimmed) return;
const exists = includedOrgs.some(
(org) => org.toLowerCase() === trimmed.toLowerCase(),
);
if (!exists) {
onGitHubConfigChange({
...githubConfig,
includeOrganizations: [...includedOrgs, trimmed],
});
}
setCustomOrgName("");
}, [customOrgName, includedOrgs, githubConfig, onGitHubConfigChange]);
const removeIncludedOrg = React.useCallback(
(name: string) => {
onGitHubConfigChange({
...githubConfig,
includeOrganizations: includedOrgs.filter(
(org) => org.toLowerCase() !== name.toLowerCase(),
),
});
},
[includedOrgs, githubConfig, onGitHubConfigChange],
);
const loadStarLists = React.useCallback(async () => {
if (
loadingStarLists ||
@@ -927,6 +960,84 @@ export function GitHubMirrorSettings({
</p>
</div>
</div>
<div className="flex items-start space-x-3">
<Checkbox
id="skip-personal-repos"
checked={advancedOptions.skipPersonalRepos ?? false}
onCheckedChange={(checked) => handleAdvancedChange('skipPersonalRepos', !!checked)}
/>
<div className="space-y-0.5 flex-1">
<Label
htmlFor="skip-personal-repos"
className="text-sm font-normal cursor-pointer flex items-center gap-2"
>
<Users className="h-3.5 w-3.5" />
Skip personal repositories (only mirror organization repos)
</Label>
<p className="text-xs text-muted-foreground">
Exclude repositories owned by your personal GitHub account; only mirror repos belonging to organizations
</p>
</div>
</div>
<div className="flex items-start space-x-3">
<Users className="h-4 w-4 mt-0.5 text-muted-foreground" />
<div className="space-y-2 flex-1">
<div className="space-y-0.5">
<Label className="text-sm font-normal flex items-center gap-2">
Limit to specific organizations
</Label>
<p className="text-xs text-muted-foreground">
Leave empty to mirror repos from every organization you belong to.
Add one or more organizations to mirror only their repositories.
</p>
</div>
{includedOrgs.length > 0 && (
<div className="flex flex-wrap gap-1.5">
{includedOrgs.map((org) => (
<Badge key={org} variant="secondary" className="gap-1">
<span>{org}</span>
<button
type="button"
onClick={() => removeIncludedOrg(org)}
className="rounded-sm hover:text-foreground/80"
aria-label={`Remove ${org} organization`}
>
<X className="h-3 w-3" />
</button>
</Badge>
))}
</div>
)}
<div className="flex items-center gap-2">
<Input
value={customOrgName}
onChange={(event) => setCustomOrgName(event.target.value)}
onKeyDown={(event) => {
if (event.key === "Enter") {
event.preventDefault();
addIncludedOrg();
}
}}
placeholder="Add organization name"
className="h-8 text-xs"
/>
<Button
type="button"
variant="outline"
size="sm"
className="h-8"
onClick={addIncludedOrg}
disabled={!customOrgName.trim()}
>
Add
</Button>
</div>
</div>
</div>
</div>
</div>
</div>
+189 -106
View File
@@ -98,6 +98,8 @@ export default function Repository() {
const [repoToDelete, setRepoToDelete] = useState<Repository | null>(null);
const [isDeleteRepoDialogOpen, setIsDeleteRepoDialogOpen] = useState(false);
const [isDeletingRepo, setIsDeletingRepo] = useState(false);
const [isBulkDeleteDialogOpen, setIsBulkDeleteDialogOpen] = useState(false);
const [isDeletingBulk, setIsDeletingBulk] = useState(false);
// Create a stable callback using useCallback
const handleNewMessage = useCallback((data: MirrorJob) => {
@@ -919,11 +921,9 @@ export default function Repository() {
setIsDeletingRepo(true);
try {
const response = await apiRequest<{ success: boolean; error?: string }>(
`/repositories/${repoToDelete.id}`,
{
method: "DELETE",
}
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [repoToDelete.id] }) }
);
if (response.success) {
@@ -941,6 +941,30 @@ export default function Repository() {
}
};
const handleBulkDelete = async () => {
if (!user || !user.id) return;
setIsDeletingBulk(true);
try {
const response = await apiRequest<{ success: boolean; deleted?: number; error?: string }>(
"/repositories",
{ method: "DELETE", body: JSON.stringify({ ids: [...selectedRepoIds] }) }
);
if (response.success) {
const count = response.deleted ?? selectedRepoIds.size;
toast.success(`Removed ${count} ${count === 1 ? "repository" : "repositories"} from Gitea Mirror.`);
setSelectedRepoIds(new Set());
await fetchRepositories(false);
} else {
showErrorToast(response.error || "Failed to delete repositories", toast);
}
} catch (error) {
showErrorToast(error, toast);
} finally {
setIsDeletingBulk(false);
setIsBulkDeleteDialogOpen(false);
}
};
// Determine what actions are available for selected repositories
const getAvailableActions = () => {
if (selectedRepoIds.size === 0) return [];
@@ -977,7 +1001,9 @@ export default function Repository() {
if (selectedRepos.some(repo => repo.status === "ignored")) {
actions.push('include');
}
actions.push('delete');
return actions;
};
@@ -994,6 +1020,7 @@ export default function Repository() {
retry: selectedRepos.filter(repo => repo.status === "failed").length,
ignore: selectedRepos.filter(repo => repo.status !== "ignored").length,
include: selectedRepos.filter(repo => repo.status === "ignored").length,
delete: selectedRepos.length,
};
};
@@ -1312,111 +1339,122 @@ export default function Repository() {
</Button>
</div>
{/* Bulk actions on desktop - integrated into the same line */}
{/* Mirror All action */}
<div className="flex items-center gap-2 border-l pl-4">
{selectedRepoIds.size === 0 ? (
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
) : (
<>
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
</>
)}
<Button
variant="default"
onClick={handleMirrorAllRepos}
disabled={isInitialLoading || loadingRepoIds.size > 0}
className="whitespace-nowrap"
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror All
</Button>
</div>
</div>
</div>
{/* Desktop: Bulk actions row - shown when repos are selected */}
{selectedRepoIds.size > 0 && (
<div className="hidden sm:flex items-center gap-2 flex-wrap">
<div className="flex items-center gap-2 px-3 py-1 bg-muted/50 rounded-md">
<span className="text-sm font-medium">
{selectedRepoIds.size} selected
</span>
<Button
variant="ghost"
size="icon"
className="h-5 w-5"
onClick={() => setSelectedRepoIds(new Set())}
>
<X className="h-3 w-3" />
</Button>
</div>
{availableActions.includes('mirror') && (
<Button
variant="default"
size="default"
onClick={handleBulkMirror}
disabled={loadingRepoIds.size > 0}
>
<FlipHorizontal className="h-4 w-4 mr-2" />
Mirror ({actionCounts.mirror})
</Button>
)}
{availableActions.includes('sync') && (
<Button
variant="outline"
size="default"
onClick={handleBulkSync}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Sync ({actionCounts.sync})
</Button>
)}
{availableActions.includes('rerun-metadata') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRerunMetadata}
disabled={loadingRepoIds.size > 0}
>
<RefreshCw className="h-4 w-4 mr-2" />
Re-run Metadata ({actionCounts.rerunMetadata})
</Button>
)}
{availableActions.includes('retry') && (
<Button
variant="outline"
size="default"
onClick={handleBulkRetry}
disabled={loadingRepoIds.size > 0}
>
<RotateCcw className="h-4 w-4 mr-2" />
Retry
</Button>
)}
{availableActions.includes('ignore') && (
<Button
variant="ghost"
size="default"
onClick={() => handleBulkSkip(true)}
disabled={loadingRepoIds.size > 0}
>
<Ban className="h-4 w-4 mr-2" />
Ignore
</Button>
)}
{availableActions.includes('include') && (
<Button
variant="outline"
size="default"
onClick={() => handleBulkSkip(false)}
disabled={loadingRepoIds.size > 0}
>
<Check className="h-4 w-4 mr-2" />
Include
</Button>
)}
<Button
variant="destructive"
size="default"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
)}
{/* Action buttons for mobile - only show when items are selected */}
{selectedRepoIds.size > 0 && (
<div className="flex items-center gap-2 flex-wrap sm:hidden">
@@ -1506,6 +1544,16 @@ export default function Repository() {
Include
</Button>
)}
<Button
variant="destructive"
size="sm"
onClick={() => setIsBulkDeleteDialogOpen(true)}
disabled={loadingRepoIds.size > 0}
>
<Trash2 className="h-4 w-4 mr-2" />
Delete ({actionCounts.delete})
</Button>
</div>
</div>
)}
@@ -1586,6 +1634,41 @@ export default function Repository() {
</DialogContent>
</Dialog>
<Dialog
open={isBulkDeleteDialogOpen}
onOpenChange={(open) => {
if (!open && !isDeletingBulk) setIsBulkDeleteDialogOpen(false);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Remove {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"} from Gitea Mirror?</DialogTitle>
<DialogDescription>
These repositories will be deleted from Gitea Mirror only. Any mirrors on Gitea will remain untouched; remove them manually in Gitea if needed.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => setIsBulkDeleteDialogOpen(false)}
disabled={isDeletingBulk}
>
Cancel
</Button>
<Button variant="destructive" onClick={handleBulkDelete} disabled={isDeletingBulk}>
{isDeletingBulk ? (
<LoaderCircle className="h-4 w-4 animate-spin" />
) : (
<span className="flex items-center gap-2">
<Trash2 className="h-4 w-4" />
Delete {selectedRepoIds.size} {selectedRepoIds.size === 1 ? "repository" : "repositories"}
</span>
)}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog
open={isDeleteRepoDialogOpen}
onOpenChange={(open) => {
+57 -12
View File
@@ -99,6 +99,7 @@ export default function RepositoryTable({
onDismissSync,
}: RepositoryTableProps) {
const tableParentRef = useRef<HTMLDivElement>(null);
const lastSelectedIndexRef = useRef<number | null>(null);
const { giteaConfig } = useGiteaConfig();
const handleUpdateDestination = async (repoId: string, newDestination: string | null) => {
@@ -235,6 +236,7 @@ export default function RepositoryTable({
// Selection handlers
const handleSelectAll = (checked: boolean) => {
lastSelectedIndexRef.current = null;
if (checked) {
const allIds = new Set(
visibleRepositories
@@ -247,7 +249,7 @@ export default function RepositoryTable({
}
};
const handleSelectRepo = (repoId: string, checked: boolean) => {
const handleSelectRepo = (repoId: string, checked: boolean, index?: number) => {
const newSelection = new Set(selectedRepoIds);
if (checked) {
newSelection.add(repoId);
@@ -255,6 +257,32 @@ export default function RepositoryTable({
newSelection.delete(repoId);
}
onSelectionChange(newSelection);
if (index !== undefined) {
lastSelectedIndexRef.current = index;
}
};
const handleShiftRangeSelect = (currentIndex: number) => {
const lastIndex = lastSelectedIndexRef.current;
if (lastIndex === null) {
const repo = visibleRepositories[currentIndex];
if (repo?.id) {
const newSelection = new Set(selectedRepoIds);
newSelection.add(repo.id);
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
}
return;
}
const start = Math.min(lastIndex, currentIndex);
const end = Math.max(lastIndex, currentIndex);
const newSelection = new Set(selectedRepoIds);
for (let i = start; i <= end; i++) {
const id = visibleRepositories[i]?.id;
if (id) newSelection.add(id);
}
onSelectionChange(newSelection);
lastSelectedIndexRef.current = currentIndex;
};
const isAllSelected =
@@ -263,7 +291,7 @@ export default function RepositoryTable({
const isPartiallySelected = selectedRepoIds.size > 0 && !isAllSelected;
// Mobile card layout for repository
const RepositoryCard = ({ repo }: { repo: Repository }) => {
const RepositoryCard = ({ repo, index }: { repo: Repository; index: number }) => {
const isLoading = repo.id ? loadingRepoIds.has(repo.id) : false;
const isSelected = repo.id ? selectedRepoIds.has(repo.id) : false;
const giteaUrl = getGiteaRepoUrl(repo);
@@ -274,12 +302,21 @@ export default function RepositoryTable({
<div className="flex flex-col gap-3">
{/* Header with checkbox and repo name */}
<div className="flex items-start gap-3">
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
<div
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(index);
}
}}
>
<Checkbox
checked={isSelected}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, checked as boolean, index)}
className="mt-1 h-5 w-5"
aria-label={`Select ${repo.name}`}
/>
</div>
<div className="flex-1 min-w-0">
<h3 className="font-medium text-base truncate">{repo.name}</h3>
<div className="flex items-center gap-2 mt-1 flex-wrap">
@@ -635,8 +672,8 @@ export default function RepositoryTable({
</div>
{/* Repository cards */}
{visibleRepositories.map((repo) => (
<RepositoryCard key={repo.id} repo={repo} />
{visibleRepositories.map((repo, index) => (
<RepositoryCard key={repo.id} repo={repo} index={index} />
))}
</div>
@@ -701,10 +738,18 @@ export default function RepositoryTable({
className="h-[65px] flex items-center justify-between bg-transparent border-b hover:bg-muted/50"
>
{/* Checkbox */}
<div className="h-full p-3 flex items-center justify-center flex-[0.3]">
<div
className="h-full p-3 flex items-center justify-center flex-[0.3]"
onClickCapture={(e) => {
if (e.shiftKey && repo.id) {
e.stopPropagation();
handleShiftRangeSelect(virtualRow.index);
}
}}
>
<Checkbox
checked={repo.id ? selectedRepoIds.has(repo.id) : false}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked)}
onCheckedChange={(checked) => repo.id && handleSelectRepo(repo.id, !!checked, virtualRow.index)}
aria-label={`Select ${repo.name}`}
/>
</div>
+3
View File
@@ -34,6 +34,7 @@ export const githubConfigSchema = z.object({
autoMirrorStarred: z.boolean().default(false),
skipStarredIssues: z.boolean().optional(), // Deprecated: kept for backward compatibility, use starredCodeOnly instead
starredDuplicateStrategy: z.enum(["suffix", "prefix", "owner-org"]).default("suffix").optional(),
skipPersonalRepos: z.boolean().default(false),
});
export const backupStrategyEnum = z.enum([
@@ -156,7 +157,9 @@ export const configSchema = z.object({
isActive: z.boolean().default(true),
githubConfig: githubConfigSchema,
giteaConfig: giteaConfigSchema,
// Unused/reserved — stored for future glob support but not currently read
include: z.array(z.string()).default(["*"]),
// Unused/reserved — stored for future glob support but not currently read
exclude: z.array(z.string()).default([]),
scheduleConfig: scheduleConfigSchema,
cleanupConfig: cleanupConfigSchema,
+13 -1
View File
@@ -20,12 +20,14 @@ interface EnvConfig {
skipForks?: boolean;
includeArchived?: boolean;
mirrorOrganizations?: boolean;
includeOrganizations?: string[];
preserveOrgStructure?: boolean;
onlyMirrorOrgs?: boolean;
starredCodeOnly?: boolean;
autoMirrorStarred?: boolean;
starredReposOrg?: string;
starredReposMode?: 'dedicated-org' | 'preserve-owner';
starredDuplicateStrategy?: 'suffix' | 'prefix' | 'owner-org';
starredLists?: string[];
mirrorStrategy?: 'preserve' | 'single-org' | 'flat-user' | 'mixed';
};
@@ -101,6 +103,9 @@ function parseEnvConfig(): EnvConfig {
const protectedRepos = process.env.CLEANUP_PROTECTED_REPOS
? process.env.CLEANUP_PROTECTED_REPOS.split(',').map(r => r.trim()).filter(Boolean)
: undefined;
const includeOrganizations = process.env.INCLUDE_ORGANIZATIONS
? process.env.INCLUDE_ORGANIZATIONS.split(',').map((org) => org.trim()).filter(Boolean)
: undefined;
const starredLists = process.env.MIRROR_STARRED_LISTS
? process.env.MIRROR_STARRED_LISTS.split(',').map((list) => list.trim()).filter(Boolean)
: undefined;
@@ -121,12 +126,14 @@ function parseEnvConfig(): EnvConfig {
skipForks: process.env.SKIP_FORKS === 'true',
includeArchived: process.env.INCLUDE_ARCHIVED === 'true',
mirrorOrganizations: process.env.MIRROR_ORGANIZATIONS === 'true',
includeOrganizations,
preserveOrgStructure: process.env.PRESERVE_ORG_STRUCTURE === 'true',
onlyMirrorOrgs: process.env.ONLY_MIRROR_ORGS === 'true',
starredCodeOnly: process.env.SKIP_STARRED_ISSUES === 'true',
autoMirrorStarred: process.env.AUTO_MIRROR_STARRED === 'true',
starredReposOrg: process.env.STARRED_REPOS_ORG,
starredReposMode: process.env.STARRED_REPOS_MODE as 'dedicated-org' | 'preserve-owner',
starredDuplicateStrategy: process.env.STARRED_DUPLICATE_STRATEGY as 'suffix' | 'prefix' | 'owner-org',
starredLists,
mirrorStrategy: process.env.MIRROR_STRATEGY as 'preserve' | 'single-org' | 'flat-user' | 'mixed',
},
@@ -277,14 +284,19 @@ export async function initializeConfigFromEnv(): Promise<void> {
includePrivate: envConfig.github.privateRepositories ?? existingConfig?.[0]?.githubConfig?.includePrivate ?? false,
includePublic: envConfig.github.publicRepositories ?? existingConfig?.[0]?.githubConfig?.includePublic ?? true,
includeCollaboratorRepos: envConfig.github.includeCollaboratorRepos ?? existingConfig?.[0]?.githubConfig?.includeCollaboratorRepos ?? true,
includeOrganizations: envConfig.github.mirrorOrganizations ? [] : (existingConfig?.[0]?.githubConfig?.includeOrganizations ?? []),
// Opt-in org allowlist from INCLUDE_ORGANIZATIONS (comma-separated). Falls
// back to existing config so the UI-managed list isn't clobbered on restart.
includeOrganizations: envConfig.github.includeOrganizations ?? existingConfig?.[0]?.githubConfig?.includeOrganizations ?? [],
starredReposOrg: envConfig.github.starredReposOrg || existingConfig?.[0]?.githubConfig?.starredReposOrg || 'starred',
starredReposMode: envConfig.github.starredReposMode || existingConfig?.[0]?.githubConfig?.starredReposMode || 'dedicated-org',
starredDuplicateStrategy: envConfig.github.starredDuplicateStrategy || existingConfig?.[0]?.githubConfig?.starredDuplicateStrategy || 'suffix',
mirrorStrategy,
defaultOrg: envConfig.gitea.organization || existingConfig?.[0]?.githubConfig?.defaultOrg || 'github-mirrors',
starredCodeOnly: envConfig.github.starredCodeOnly ?? existingConfig?.[0]?.githubConfig?.starredCodeOnly ?? false,
autoMirrorStarred: envConfig.github.autoMirrorStarred ?? existingConfig?.[0]?.githubConfig?.autoMirrorStarred ?? false,
starredLists: envConfig.github.starredLists ?? existingConfig?.[0]?.githubConfig?.starredLists ?? [],
// ONLY_MIRROR_ORGS=true maps to skipPersonalRepos: true
skipPersonalRepos: envConfig.github.onlyMirrorOrgs ?? existingConfig?.[0]?.githubConfig?.skipPersonalRepos ?? false,
};
// Build Gitea config
+38 -4
View File
@@ -43,13 +43,18 @@ type SyncDependencies = {
/**
* Enhanced repository information including mirror status
*/
interface GiteaRepoInfo {
export interface GiteaRepoInfo {
id: number;
name: string;
owner: { login: string } | string;
mirror: boolean;
mirror_interval?: string;
clone_url?: string;
// Original migration source URL. Gitea/Forgejo populate this with the
// upstream clone address for migrated/mirrored repos, so it is the
// authoritative way to tell whether an existing mirror points at THIS
// GitHub source (vs. a same-named mirror of a different source).
original_url?: string;
private: boolean;
}
@@ -544,9 +549,12 @@ export async function syncGiteaRepoEnhanced({
// Create backup if strategy says so
if (shouldBackupForStrategy(backupStrategy, forcePushDetected)) {
const cloneUrl =
repoInfo.clone_url ||
`${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
// Always derive the clone URL from the user-configured Gitea URL rather
// than repoInfo.clone_url (which reflects Gitea's ROOT_URL and may be
// unreachable from the app — e.g. Tailscale MagicDNS deployments where
// ROOT_URL resolves externally but the app talks to Gitea on a private
// address).
const cloneUrl = `${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
try {
const backupResult = await createPreSyncBundleBackup({
@@ -886,6 +894,32 @@ export async function syncGiteaRepoEnhanced({
status: "failed",
});
}
} else if (syncError instanceof HttpError && syncError.status === 405) {
// Gitea returns HTTP 405 (with an empty body) when the repository is not
// a pull-mirror in its database — e.g. Gitea auto-disabled the mirror or
// the repo lost its mirror state after a manual edit.
const actionableMessage =
`Gitea reports this repository is not a pull mirror (HTTP 405). ` +
`In Gitea check Settings → Mirror Settings; if the mirror section is ` +
`missing, delete the repository in Gitea and re-mirror it from gitea-mirror.`;
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("failed"),
updatedAt: new Date(),
errorMessage: actionableMessage,
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Sync failed: ${repository.name} is not a pull mirror in Gitea (HTTP 405)`,
details: actionableMessage,
status: "failed",
});
}
throw syncError;
}
+232
View File
@@ -0,0 +1,232 @@
/**
* Unit tests for release reconciliation logic regression for #310.
*
* Root-cause verdict: Theory A (not Theory B).
*
* The old `needsRecreation` check compared published_at-based expected indices
* against Gitea's API order (which mirrors sort by tag-commit-date, not
* published_at). For repos where published_at order permanently disagrees with
* tag-commit-date order, `currentExpectedIdx < nextExpectedIdx` evaluates true
* on every single sync, triggering delete-and-recreate forever.
*
* Discriminating evidence unaconfig_dart fixture:
* v0.1.1 published_at 2024-01-13T23:44 (earlier) expectedOrder index 0
* v0.1.0 published_at 2024-01-16T00:23 (later) expectedOrder index 1
* v0.1.0 tagged 2024-01-13T23:30 (earlier tag commit)
* v0.1.1 tagged 2024-01-13T23:42 (later tag commit)
* Gitea tag-commit order: [v0.1.1, v0.1.0] (v0.1.1 has newer commit)
* Old check: currentExpectedIdx(v0.1.1)=0 < nextExpectedIdx(v0.1.0)=1 TRUE
* needsRecreation fires on every sync, forever.
*
* Theory B (operator inversion) would fire for ALL repos with >1 release, but
* field evidence shows only ~2 of 150 repos are affected ruling it out.
*
* Fix: replaced `needsRecreation` machinery with set-based reconciliation via
* `classifyReleasesForReconciliation`. Releases are created when missing, skipped
* (or PATCH-updated if content drifted) when present. No deletions for ordering.
*/
import { describe, expect, it } from "bun:test";
import {
classifyReleasesForReconciliation,
classifyAssetsForReconciliation,
} from "@/lib/gitea";
describe("classifyReleasesForReconciliation", () => {
describe("normal repo — published_at order matches tag-commit order", () => {
it("creates releases missing in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
expect(toSkip).toEqual([]);
});
it("skips releases already present in Gitea", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.1.0", "v1.2.0"];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0", "v1.1.0", "v1.2.0"]);
});
it("creates missing releases while skipping existing ones", () => {
const github = ["v1.0.0", "v1.1.0", "v1.2.0"];
const gitea = ["v1.0.0", "v1.2.0"]; // v1.1.0 is missing
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v1.1.0"]);
expect(toSkip).toEqual(["v1.0.0", "v1.2.0"]);
});
it("does NOT produce any deletions — order mismatches are ignored", () => {
// Even if Gitea has them in a different order, the function never suggests deletion
const github = ["v1.0.0", "v1.1.0"];
const gitea = ["v1.1.0", "v1.0.0"]; // reversed order from Gitea
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v1.0.0");
expect(toSkip).toContain("v1.1.0");
});
});
describe("unaconfig_dart regression — published_at order disagrees with tag-commit order (#310)", () => {
// v0.1.0: tagged 2024-01-13T23:30, published_at 2024-01-16T00:23 (published AFTER v0.1.1)
// v0.1.1: tagged 2024-01-13T23:42, published_at 2024-01-13T23:44 (published BEFORE v0.1.0)
//
// Gitea display order (by tag-commit date): [v0.1.1, v0.1.0] (v0.1.1 tagged later)
// GitHub published_at order (oldest first): [v0.1.1, v0.1.0] (v0.1.1 published earlier)
// Wait — in this specific case the orders AGREE. The inversion scenario is:
// GitHub sorts descending (newest first): [v0.1.0, v0.1.1]
// Gitea API returns by tag-commit DESC: [v0.1.1, v0.1.0]
// Old expectedOrder (ascending published): v0.1.1→0, v0.1.0→1
// Check for [v0.1.1, v0.1.0]: current=v0.1.1(idx=0) < next=v0.1.0(idx=1) → TRUE every time
it("does NOT trigger recreation when published_at order and tag-commit order disagree", () => {
// Both releases already in Gitea (as they would be after first successful sync).
// Old code would fire needsRecreation=true here on every subsequent sync.
// New code: set-based check — both present → toCreate is empty → no deletions.
const github = ["v0.1.0", "v0.1.1"]; // GitHub API returns newest published_at first
const gitea = ["v0.1.1", "v0.1.0"]; // Gitea tag-commit order (v0.1.1 tagged later)
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]); // nothing to create
expect(toSkip).toHaveLength(2);
expect(toSkip).toContain("v0.1.0");
expect(toSkip).toContain("v0.1.1");
});
it("creates v0.1.0 and v0.1.1 when Gitea has no releases yet (first sync)", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea: string[] = [];
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0", "v0.1.1"]);
expect(toSkip).toEqual([]);
});
it("only creates the missing release when one of the two already exists", () => {
const github = ["v0.1.0", "v0.1.1"];
const gitea = ["v0.1.1"]; // only v0.1.1 was created so far
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual(["v0.1.0"]);
expect(toSkip).toEqual(["v0.1.1"]);
});
});
describe("edge cases", () => {
it("handles empty GitHub releases list", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], ["v1.0.0"]);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("handles both lists empty", () => {
const { toCreate, toSkip } = classifyReleasesForReconciliation([], []);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual([]);
});
it("ignores Gitea releases that are not in the GitHub set (orphans, handled by retention cleanup)", () => {
const github = ["v1.0.0"];
const gitea = ["v1.0.0", "v0.9.0"]; // v0.9.0 is an orphan not in GitHub's limited set
const { toCreate, toSkip } = classifyReleasesForReconciliation(github, gitea);
expect(toCreate).toEqual([]);
expect(toSkip).toEqual(["v1.0.0"]);
// v0.9.0 not mentioned in either output — handled by retention cleanup, not here
});
});
});
/**
* Asset reconciliation regression for #331.
*
* Root cause: assets were uploaded only on the create path. When a Gitea release
* already existed (every re-sync, or after an interrupted first upload), the update
* path PATCHed the body and `continue`d without ever touching assets so a release
* that existed without its full asset set stayed permanently asset-less and re-syncing
* could never heal it. Reproduced on a real Forgejo pull-mirror: GitHub release with
* two 35-40MB binaries Gitea release with 0 assets re-sync logged "Updating
* existing release" and left it at 0.
*
* Fix: reconcile assets idempotently on both paths via classifyAssetsForReconciliation.
*/
describe("classifyAssetsForReconciliation", () => {
it("uploads all assets when the Gitea release has none (the #331 broken state)", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea: Array<{ id: number; name: string; size: number }> = [];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([
{ name: "base.zip", replaceAssetId: null },
{ name: "extras.zip", replaceAssetId: null },
]);
});
it("backfills only the missing asset when one already exists", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [{ id: 9, name: "base.zip", size: 40_264_954 }];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual(["base.zip"]);
expect(toUpload).toEqual([{ name: "extras.zip", replaceAssetId: null }]);
});
it("is idempotent — skips everything when all assets already match by name+size", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [
{ id: 9, name: "base.zip", size: 40_264_954 },
{ id: 10, name: "extras.zip", size: 37_098_528 },
];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual(["base.zip", "extras.zip"]);
});
it("replaces an asset whose size changed upstream (re-upload over the stale copy)", () => {
const github = [{ name: "firmware.bin", size: 2048 }];
const gitea = [{ id: 42, name: "firmware.bin", size: 1024 }]; // truncated/stale
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([{ name: "firmware.bin", replaceAssetId: 42 }]);
});
it("handles a release with no GitHub assets", () => {
const { toUpload, toSkip } = classifyAssetsForReconciliation(
[],
[{ id: 1, name: "leftover.zip", size: 10 }]
);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual([]);
});
});
+517 -243
View File
@@ -579,7 +579,27 @@ export const mirrorGithubRepoToGitea = async ({
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
if (
// REUSE-FIRST (issues #315 / #309): before generating any (suffixed) name,
// check whether this exact source is already mirrored — either at the
// recorded mirroredLocation or at the base name. If so, reuse that location
// and route into the "already mirrored" handling below instead of creating
// a duplicate. This must run before generateUniqueRepoName so the names
// converge under concurrency (the in-flight guard then becomes effective).
const { findExistingMirror } = await import("./utils/mirror-source-match");
const existingMirror = await findExistingMirror({
repository,
config,
candidateOwner: repoOwner,
candidateName: repository.name,
});
if (existingMirror) {
repoOwner = existingMirror.owner;
targetRepoName = existingMirror.repoName;
console.log(
`Reusing existing same-source mirror for ${repository.fullName} at ${repoOwner}/${targetRepoName}`
);
} else if (
repository.isStarred &&
config.githubConfig &&
(config.githubConfig.starredReposMode || "dedicated-org") === "dedicated-org"
@@ -594,6 +614,7 @@ export const mirrorGithubRepoToGitea = async ({
githubOwner,
fullName: repository.fullName,
strategy: config.githubConfig.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
if (targetRepoName !== repository.name) {
@@ -643,45 +664,72 @@ export const mirrorGithubRepoToGitea = async ({
strategy: "delete", // Can be configured: "skip", "delete", or "rename"
});
} else if (existingRepoInfo?.mirror) {
console.log(
`Repository ${targetRepoName} already exists in Gitea under ${repoOwner}. Updating database status.`
);
// PHANTOM-FORK GUARD (#309): a mirror at this name is only "ours" if it
// mirrors THIS source. existingMirror short-circuits the check
// because findExistingMirror already confirmed the source match.
const { isMirrorOfSource } = await import("./utils/mirror-source-match");
const sameSource =
!!existingMirror ||
isMirrorOfSource(existingRepoInfo, repository.cloneUrl);
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
if (!sameSource) {
// A different source occupies this name. Treat as a genuine collision:
// generate a unique name and fall through to create a separate mirror.
console.warn(
`[Mirror] ${repoOwner}/${targetRepoName} is a mirror of a different source. ` +
`Generating a unique name for ${repository.fullName} to avoid overwriting it.`
);
targetRepoName = await generateUniqueRepoName({
config,
orgName: repoOwner,
baseName: repository.name,
githubOwner: repository.fullName.split("/")[0],
fullName: repository.fullName,
strategy: config.githubConfig?.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
// expectedLocation is recomputed below before the "mirroring" write.
} else {
console.log(
`Repository ${targetRepoName} already exists in Gitea under ${repoOwner}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
// Append log for "mirrored" status
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea`,
details: `Repository ${repository.name} was found to already exist in Gitea under ${repoOwner} and database status was updated.`,
status: "mirrored",
});
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
console.log(
`Repository ${repository.name} database status updated to mirrored`
);
return;
// Append log for "mirrored" status
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea`,
details: `Repository ${repository.name} was found to already exist in Gitea under ${repoOwner} and database status was updated.`,
status: "mirrored",
});
console.log(
`Repository ${repository.name} database status updated to mirrored`
);
return;
}
} else {
console.warn(
`[Mirror] Repository ${repoOwner}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
@@ -689,6 +737,10 @@ export const mirrorGithubRepoToGitea = async ({
}
}
// Recompute the target location in case a phantom-fork collision above
// forced a renamed target after the initial expectedLocation was derived.
const targetLocation = `${repoOwner}/${targetRepoName}`;
console.log(`Mirroring repository ${repository.name}`);
// DOUBLE-CHECK: Final idempotency check right before updating status
@@ -696,7 +748,7 @@ export const mirrorGithubRepoToGitea = async ({
const finalCheck = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
expectedLocation,
expectedLocation: targetLocation,
});
if (finalCheck) {
@@ -714,7 +766,7 @@ export const mirrorGithubRepoToGitea = async ({
.update(repositories)
.set({
status: repoStatusEnum.parse("mirroring"),
mirroredLocation: expectedLocation,
mirroredLocation: targetLocation,
updatedAt: new Date(),
})
.where(eq(repositories.id, repository.id!));
@@ -1177,6 +1229,14 @@ async function isMirroredLocationClaimedInDb({
* Checks both the Gitea instance (HTTP) and the local DB (mirroredLocation)
* to reduce collisions during concurrent batch mirroring.
*
* Source-aware (issues #315 / #309): when a candidate name is already occupied
* by a mirror of THIS SAME GitHub source, the name is REUSED rather than
* suffixed this is what previously caused starred repos to spawn `-owner`,
* `-owner-1`, duplicates on every re-mirror. Suffixing only happens on a
* genuine different-source collision (preserving the #95/#236 cross-owner
* behavior). The per-user DB claim check is retained so two users mirroring the
* same source into a shared org stay separated.
*
* NOTE: This function only checks availability it does NOT claim the name.
* The actual claim happens later when mirroredLocation is written at the
* status="mirroring" DB update, which is protected by a unique partial index
@@ -1189,6 +1249,7 @@ async function generateUniqueRepoName({
githubOwner,
fullName,
strategy,
sourceCloneUrl,
}: {
config: Partial<Config>;
orgName: string;
@@ -1196,6 +1257,10 @@ async function generateUniqueRepoName({
githubOwner: string;
fullName: string;
strategy?: string;
// Source GitHub clone URL, used to decide whether an occupied name belongs to
// THIS repo's mirror (reuse) or a different source (suffix). When omitted,
// behavior degrades to the legacy "any occupant collides" semantics.
sourceCloneUrl?: string;
}): Promise<string> {
if (!fullName?.includes("/")) {
throw new Error(
@@ -1206,33 +1271,55 @@ async function generateUniqueRepoName({
const duplicateStrategy = strategy || "suffix";
const userId = config.userId || "";
// Helper: check both Gitea and local DB for a candidate name
const isNameTaken = async (candidateName: string): Promise<boolean> => {
const { getGiteaRepoInfo } = await import("./gitea-enhanced");
const { classifyCandidateName } = await import("./utils/mirror-source-match");
// Resolve the I/O for a candidate name (Gitea existence, DB claim, repo info)
// and defer the available/reusable/taken decision to the pure, unit-tested
// classifyCandidateName helper.
const classifyName = async (candidateName: string) => {
const existsInGitea = await isRepoPresentInGitea({
config,
owner: orgName,
repoName: candidateName,
});
if (existsInGitea) return true;
// Also check local DB to catch concurrent batch operations
// where another repo claimed this location but hasn't created it in Gitea yet
// A DB claim by a DIFFERENT repo (concurrent batch) always blocks reuse.
let claimedByOther = false;
if (userId) {
const claimedInDb = await isMirroredLocationClaimedInDb({
claimedByOther = await isMirroredLocationClaimedInDb({
userId,
candidateLocation: `${orgName}/${candidateName}`,
excludeFullName: fullName,
});
if (claimedInDb) return true;
}
return false;
// Only fetch repo info when it can actually change the decision (existing,
// same-source candidate that is not DB-claimed by another repo).
const repoInfo =
existsInGitea && sourceCloneUrl && !claimedByOther
? await getGiteaRepoInfo({
config,
owner: orgName,
repoName: candidateName,
})
: null;
return classifyCandidateName({
existsInGitea,
claimedByOther,
repoInfo,
sourceCloneUrl,
});
};
// First check if base name is available
const baseExists = await isNameTaken(baseName);
if (!baseExists) {
// First check the base name — reuse it if it already holds our own mirror.
const baseClass = await classifyName(baseName);
if (baseClass === "available") {
return baseName;
}
if (baseClass === "reusable") {
console.log(`Reusing existing same-source mirror name: ${orgName}/${baseName}`);
return baseName;
}
@@ -1262,9 +1349,14 @@ async function generateUniqueRepoName({
break;
}
const exists = await isNameTaken(candidateName);
const candidateClass = await classifyName(candidateName);
if (!exists) {
if (candidateClass === "reusable") {
console.log(`Reusing existing same-source mirror name: ${orgName}/${candidateName}`);
return candidateName;
}
if (candidateClass === "available") {
console.log(`Found unique name for duplicate starred repo: ${candidateName}`);
return candidateName;
}
@@ -1314,8 +1406,29 @@ export async function mirrorGitHubRepoToGiteaOrg({
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
// The org we will record/reuse for. Stays === orgName on the create path
// (migration uses orgName + giteaOrgId); a reuse hit may repoint it to the
// recorded mirroredLocation's owner for the early-return DB update.
let targetOwner = orgName;
if (
// REUSE-FIRST (issues #315 / #309): reuse an existing same-source mirror
// before generating any suffixed name. See mirrorGithubRepoToGitea for the
// rationale. Routes a hit into the "already mirrored" handling below.
const { findExistingMirror } = await import("./utils/mirror-source-match");
const existingMirror = await findExistingMirror({
repository,
config,
candidateOwner: orgName,
candidateName: repository.name,
});
if (existingMirror) {
targetOwner = existingMirror.owner;
targetRepoName = existingMirror.repoName;
console.log(
`Reusing existing same-source mirror for ${repository.fullName} at ${targetOwner}/${targetRepoName}`
);
} else if (
repository.isStarred &&
config.githubConfig &&
(config.githubConfig.starredReposMode || "dedicated-org") === "dedicated-org"
@@ -1330,6 +1443,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
githubOwner,
fullName: repository.fullName,
strategy: config.githubConfig.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
if (targetRepoName !== repository.name) {
@@ -1340,7 +1454,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
}
// IDEMPOTENCY CHECK: Check if this repo is already being mirrored
const expectedLocation = `${orgName}/${targetRepoName}`;
const expectedLocation = `${targetOwner}/${targetRepoName}`;
const isCurrentlyMirroring = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
@@ -1358,7 +1472,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const isExisting = await isRepoPresentInGitea({
config,
owner: orgName,
owner: targetOwner,
repoName: targetRepoName,
});
@@ -1366,7 +1480,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const { getGiteaRepoInfo, handleExistingNonMirrorRepo } = await import("./gitea-enhanced");
const existingRepoInfo = await getGiteaRepoInfo({
config,
owner: orgName,
owner: targetOwner,
repoName: targetRepoName,
});
@@ -1379,52 +1493,83 @@ export async function mirrorGitHubRepoToGiteaOrg({
strategy: "delete", // Can be configured: "skip", "delete", or "rename"
});
} else if (existingRepoInfo?.mirror) {
console.log(
`Repository ${targetRepoName} already exists in Gitea organization ${orgName}. Updating database status.`
);
// PHANTOM-FORK GUARD (#309): only treat this as "ours" if it mirrors
// THIS source. existingMirror short-circuits because findExistingMirror already
// confirmed the source match.
const { isMirrorOfSource } = await import("./utils/mirror-source-match");
const sameSource =
!!existingMirror ||
isMirrorOfSource(existingRepoInfo, repository.cloneUrl);
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
if (!sameSource) {
// Different source occupies this name: generate a unique name and
// fall through to create a separate mirror under orgName/giteaOrgId.
console.warn(
`[Mirror] ${targetOwner}/${targetRepoName} is a mirror of a different source. ` +
`Generating a unique name for ${repository.fullName} to avoid overwriting it.`
);
targetOwner = orgName;
targetRepoName = await generateUniqueRepoName({
config,
orgName,
baseName: repository.name,
githubOwner: repository.fullName.split("/")[0],
fullName: repository.fullName,
strategy: config.githubConfig?.starredDuplicateStrategy,
sourceCloneUrl: repository.cloneUrl,
});
} else {
console.log(
`Repository ${targetRepoName} already exists in Gitea organization ${targetOwner}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${orgName}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
await syncRepositoryMetadataToGitea({
config,
octokit,
repository,
giteaOwner: targetOwner,
giteaRepoName: targetRepoName,
giteaToken: decryptedConfig.giteaConfig.token,
});
// Create a mirror job log entry
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${targetRepoName} already exists in Gitea organization ${orgName}`,
details: `Repository ${targetRepoName} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
status: "mirrored",
});
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${targetOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
console.log(
`Repository ${targetRepoName} database status updated to mirrored in organization ${orgName}`
);
return;
// Create a mirror job log entry
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${targetRepoName} already exists in Gitea organization ${targetOwner}`,
details: `Repository ${targetRepoName} was found to already exist in Gitea organization ${targetOwner} and database status was updated.`,
status: "mirrored",
});
console.log(
`Repository ${targetRepoName} database status updated to mirrored in organization ${targetOwner}`
);
return;
}
} else {
console.warn(
`[Mirror] Repository ${orgName}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
`[Mirror] Repository ${targetOwner}/${targetRepoName} exists but mirror status could not be verified. Continuing with mirror creation flow.`
);
}
}
// Recompute the target location in case a phantom-fork collision above
// forced a renamed target after the initial expectedLocation was derived.
const targetLocation = `${orgName}/${targetRepoName}`;
console.log(
`Mirroring repository ${repository.fullName} to organization ${orgName} as ${targetRepoName}`
);
@@ -1437,7 +1582,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
const finalCheck = await isRepoCurrentlyMirroring({
config,
repoName: targetRepoName,
expectedLocation,
expectedLocation: targetLocation,
});
if (finalCheck) {
@@ -1455,7 +1600,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
.update(repositories)
.set({
status: repoStatusEnum.parse("mirroring"),
mirroredLocation: expectedLocation,
mirroredLocation: targetLocation,
updatedAt: new Date(),
})
.where(eq(repositories.id, repository.id!));
@@ -2515,6 +2660,199 @@ export const mirrorGitRepoIssuesToGitea = async ({
);
};
/**
* Classify a set of GitHub releases against the set already present in Gitea.
*
* Returns:
* - `toCreate`: tag names that exist on GitHub but are missing from Gitea
* - `toSkip`: tag names that already exist in Gitea (will be handled by PATCH-if-content-changed)
*
* Deliberately does NOT return anything to delete based on ordering Gitea mirrors
* order releases by tag-commit date, which can permanently disagree with GitHub's
* published_at order (e.g. unaconfig_dart v0.1.0/v0.1.1 #310). Destroying and
* re-emitting releases for a cosmetic display-order difference is never worth it.
*/
export function classifyReleasesForReconciliation(
githubTagNames: string[],
giteaTagNames: string[]
): { toCreate: string[]; toSkip: string[] } {
const giteaSet = new Set(giteaTagNames);
const toCreate: string[] = [];
const toSkip: string[] = [];
for (const tag of githubTagNames) {
if (giteaSet.has(tag)) {
toSkip.push(tag);
} else {
toCreate.push(tag);
}
}
return { toCreate, toSkip };
}
/**
* Decide which of a GitHub release's assets need (re)uploading to Gitea.
*
* Compared by name:
* - present in Gitea with a matching size -> skip (already mirrored)
* - present with a different size -> upload, replacing the stale copy
* - absent -> upload (fresh)
*
* Pure function so the create/update reconciliation decision is unit-testable
* without hitting the network (regression guard for #331).
*/
export function classifyAssetsForReconciliation(
githubAssets: Array<{ name: string; size: number }>,
giteaAssets: Array<{ id: number; name: string; size: number }>
): {
toUpload: Array<{ name: string; replaceAssetId: number | null }>;
toSkip: string[];
} {
const existingByName = new Map(giteaAssets.map((a) => [a.name, a]));
const toUpload: Array<{ name: string; replaceAssetId: number | null }> = [];
const toSkip: string[] = [];
for (const asset of githubAssets) {
const existing = existingByName.get(asset.name);
if (existing && existing.size === asset.size) {
toSkip.push(asset.name);
} else {
toUpload.push({ name: asset.name, replaceAssetId: existing ? existing.id : null });
}
}
return { toUpload, toSkip };
}
/**
* Idempotently mirror a GitHub release's assets onto the matching Gitea release.
*
* Runs on BOTH the create and update paths so assets are reconciled on every sync,
* not only on the single sync where the Gitea release is first created. Previously
* assets were uploaded inline in the create path only; the update path PATCHed the
* body and `continue`d without ever looking at assets. Any release whose assets
* failed or were interrupted on first creation therefore stayed permanently
* asset-less, and re-syncing could never heal it (#331).
*
* Strategy: compare by asset name. Skip assets already present with a matching size;
* (re)upload anything missing, and replace an existing asset whose size differs
* (truncated/changed upstream). Returns per-release counts so the caller can report.
*/
async function reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId,
githubAssets,
tagName,
}: {
config: Partial<Config>;
decryptedConfig: Config;
repoOwner: string;
repoName: string;
giteaReleaseId: number;
githubAssets: Array<{ name: string; size: number; browser_download_url: string }>;
tagName: string;
}): Promise<{ uploaded: number; failed: number; skipped: number }> {
let uploaded = 0;
let failed = 0;
let skipped = 0;
if (!githubAssets || githubAssets.length === 0) {
return { uploaded, failed, skipped };
}
const giteaBaseUrl = config.giteaConfig!.url;
const giteaAuth = { Authorization: `token ${decryptedConfig.giteaConfig!.token}` };
// Fetch existing attachments so we only transfer what's missing or changed.
const existingAssets: Array<{ id: number; name: string; size: number }> = await httpGet(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets`,
giteaAuth
)
.then((r) => (Array.isArray(r?.data) ? r.data : []))
.catch(() => []);
const { toUpload, toSkip } = classifyAssetsForReconciliation(
githubAssets,
existingAssets
);
skipped = toSkip.length;
const githubByName = new Map(githubAssets.map((a) => [a.name, a]));
for (const { name, replaceAssetId } of toUpload) {
const asset = githubByName.get(name)!;
try {
// Download from GitHub. fetch strips the Authorization header on the
// cross-host redirect to GitHub's object storage, so this works for both
// public and private release assets.
console.log(
`[Releases] Downloading asset: ${asset.name} (${asset.size} bytes) for ${tagName}`
);
const assetResponse = await fetch(asset.browser_download_url, {
headers: {
Accept: "application/octet-stream",
Authorization: `token ${decryptedConfig.githubConfig!.token}`,
},
});
if (!assetResponse.ok) {
console.error(
`[Releases] Failed to download asset ${asset.name}: ${assetResponse.status} ${assetResponse.statusText}`
);
failed++;
continue;
}
const assetData = await assetResponse.arrayBuffer();
// Gitea rejects a duplicate attachment name, so drop a stale/mismatched
// copy before re-uploading.
if (replaceAssetId !== null) {
await httpDelete(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets/${replaceAssetId}`,
giteaAuth
).catch(() => null);
}
const formData = new FormData();
formData.append("attachment", new Blob([assetData]), asset.name);
const uploadResponse = await fetch(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets?name=${encodeURIComponent(asset.name)}`,
{
method: "POST",
headers: { Authorization: `token ${decryptedConfig.giteaConfig!.token}` },
body: formData,
}
);
if (uploadResponse.ok) {
console.log(`[Releases] Successfully uploaded asset: ${asset.name}`);
uploaded++;
} else {
const errorText = await uploadResponse.text();
console.error(
`[Releases] Failed to upload asset ${asset.name}: ${uploadResponse.status} ${errorText}`
);
failed++;
}
} catch (assetError) {
console.error(
`[Releases] Error processing asset ${asset.name}: ${
assetError instanceof Error ? assetError.message : String(assetError)
}`
);
failed++;
}
}
return { uploaded, failed, skipped };
}
export async function mirrorGitHubReleasesToGitea({
octokit,
repository,
@@ -2600,24 +2938,14 @@ export async function mirrorGitHubReleasesToGitea({
let mirroredCount = 0;
let skippedCount = 0;
let skippedMissingTagCount = 0;
let totalAssetsUploaded = 0;
let totalAssetsFailed = 0;
const getReleaseTimestamp = (release: (typeof limitedReleases)[number]) => {
// Use published_at first (when the release was published on GitHub)
// Fall back to created_at (when the git tag was created) only if published_at is missing
// This matches GitHub's sorting behavior and handles cases where multiple tags
// point to the same commit but have different publish dates
const sourceDate = release.published_at ?? release.created_at ?? "";
const timestamp = sourceDate ? new Date(sourceDate).getTime() : 0;
return Number.isFinite(timestamp) ? timestamp : 0;
};
// Process releases in their GitHub API order (newest first by default)
const releasesToProcess = limitedReleases.slice();
// Capture the latest releases, then process them oldest-to-newest so Gitea mirrors keep chronological order
const releasesToProcess = limitedReleases
.slice()
.sort((a, b) => getReleaseTimestamp(b) - getReleaseTimestamp(a))
.sort((a, b) => getReleaseTimestamp(a) - getReleaseTimestamp(b));
console.log(`[Releases] Processing ${releasesToProcess.length} releases in chronological order (oldest to newest by published date)`);
console.log(`[Releases] Processing ${releasesToProcess.length} releases for ${repository.fullName}`);
releasesToProcess.forEach((rel, idx) => {
const publishedDate = new Date(rel.published_at || rel.created_at);
const createdDate = new Date(rel.created_at);
@@ -2627,85 +2955,10 @@ export async function mirrorGitHubReleasesToGitea({
console.log(`[Releases] ${idx + 1}. ${rel.tag_name} - ${dateInfo}`);
});
// Check if existing releases in Gitea are in the wrong order
// If so, we need to delete and recreate them to fix the ordering
let needsRecreation = false;
try {
const existingReleasesResponse = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases?per_page=100`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
).catch(() => null);
if (existingReleasesResponse && existingReleasesResponse.data && Array.isArray(existingReleasesResponse.data)) {
const existingReleases = existingReleasesResponse.data;
if (existingReleases.length > 0) {
console.log(`[Releases] Found ${existingReleases.length} existing releases in Gitea, checking chronological order...`);
// Create a map of tag_name to expected chronological index (0 = oldest, n = newest)
const expectedOrder = new Map<string, number>();
releasesToProcess.forEach((rel, idx) => {
expectedOrder.set(rel.tag_name, idx);
});
// Check if existing releases are in the correct order based on created_unix
// Gitea sorts by created_unix DESC, so newer releases should have higher created_unix values
const releasesThatShouldExist = existingReleases.filter(r => expectedOrder.has(r.tag_name));
if (releasesThatShouldExist.length > 1) {
for (let i = 0; i < releasesThatShouldExist.length - 1; i++) {
const current = releasesThatShouldExist[i];
const next = releasesThatShouldExist[i + 1];
const currentExpectedIdx = expectedOrder.get(current.tag_name)!;
const nextExpectedIdx = expectedOrder.get(next.tag_name)!;
// Since Gitea returns releases sorted by created_unix DESC:
// - Earlier releases in the list should have HIGHER expected indices (newer)
// - Later releases in the list should have LOWER expected indices (older)
if (currentExpectedIdx < nextExpectedIdx) {
console.log(`[Releases] ⚠️ Incorrect ordering detected: ${current.tag_name} (index ${currentExpectedIdx}) appears before ${next.tag_name} (index ${nextExpectedIdx})`);
needsRecreation = true;
break;
}
}
}
if (needsRecreation) {
console.log(`[Releases] ⚠️ Releases are in incorrect chronological order. Will delete and recreate all releases.`);
// Delete all existing releases that we're about to recreate
for (const existingRelease of releasesThatShouldExist) {
try {
console.log(`[Releases] Deleting incorrectly ordered release: ${existingRelease.tag_name}`);
await httpDelete(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
);
} catch (deleteError) {
console.error(`[Releases] Failed to delete release ${existingRelease.tag_name}: ${deleteError instanceof Error ? deleteError.message : String(deleteError)}`);
}
}
console.log(`[Releases] ✅ Deleted ${releasesThatShouldExist.length} releases. Will recreate in correct chronological order.`);
} else {
console.log(`[Releases] ✅ Existing releases are in correct chronological order.`);
}
}
}
} catch (orderCheckError) {
console.warn(`[Releases] Could not verify release order: ${orderCheckError instanceof Error ? orderCheckError.message : String(orderCheckError)}`);
// Continue with normal processing
}
for (const release of releasesToProcess) {
try {
// Check if release already exists (skip check if we just deleted all releases)
const existingReleasesResponse = needsRecreation ? null : await httpGet(
// Always check if release already exists — reconcile by tag set, not by ordering
const existingReleasesResponse = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/tags/${release.tag_name}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
@@ -2744,7 +2997,8 @@ export async function mirrorGitHubReleasesToGitea({
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
{
tag_name: release.tag_name,
target: release.target_commitish,
// Omit `target` — the release already exists and is anchored to its tag;
// re-sending target_commitish risks the same "target not found" 404 (#331).
title: release.name || release.tag_name,
body: releaseNote,
draft: release.draft,
@@ -2765,6 +3019,49 @@ export async function mirrorGitHubReleasesToGitea({
console.log(`[Releases] Release ${release.tag_name} already up-to-date, skipping`);
skippedCount++;
}
// Reconcile assets on every sync — backfill any that are missing or changed.
// The update path used to `continue` here without touching assets, so a
// release that existed without its full asset set stayed broken forever (#331).
const assetResult = await reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId: existingRelease.id,
githubAssets: release.assets || [],
tagName: release.tag_name,
});
if (assetResult.uploaded > 0) {
console.log(
`[Releases] Backfilled ${assetResult.uploaded} missing/changed asset(s) for existing release ${release.tag_name}`
);
}
totalAssetsUploaded += assetResult.uploaded;
totalAssetsFailed += assetResult.failed;
continue;
}
// The git tag must already exist in Gitea before we create a release for it.
// For a mirror, tags are synced from upstream by Gitea's own git mirror, which
// can lag behind this metadata sync (e.g. a large/slow initial clone). If the
// tag isn't present yet, skip and let a later sync pick it up — do NOT ask Gitea
// to create the release against a `target` branch:
// - if the target can't be resolved Gitea returns 404 "The target couldn't be
// found" and the release is lost (#331),
// - if it can, Gitea would create a brand-new tag at the wrong commit.
const tagExists = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/tags/${encodeURIComponent(release.tag_name)}`,
{ Authorization: `token ${decryptedConfig.giteaConfig.token}` }
)
.then(() => true)
.catch(() => false);
if (!tagExists) {
console.warn(
`[Releases] Tag ${release.tag_name} is not present in Gitea yet — skipping release for now (the git mirror may still be syncing; it will be retried on the next sync)`
);
skippedMissingTagCount++;
continue;
}
@@ -2774,12 +3071,14 @@ export async function mirrorGitHubReleasesToGitea({
} else {
console.log(`[Releases] Creating release ${release.tag_name} with GitHub date header (no changelog)`);
}
const createReleaseResponse = await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases`,
{
tag_name: release.tag_name,
target: release.target_commitish,
// Intentionally omit `target`: the tag already exists (verified above), so
// Gitea attaches the release to it. Sending target_commitish can 404 with
// "The target couldn't be found" on some Gitea/Forgejo versions (#331).
title: release.name || release.tag_name,
body: releaseNote,
draft: release.draft,
@@ -2790,70 +3089,45 @@ export async function mirrorGitHubReleasesToGitea({
}
);
// Mirror release assets if they exist
// Mirror release assets if they exist (idempotent — see reconcileReleaseAssets)
if (release.assets && release.assets.length > 0) {
console.log(`[Releases] Mirroring ${release.assets.length} assets for release ${release.tag_name}`);
for (const asset of release.assets) {
try {
// Download the asset from GitHub
console.log(`[Releases] Downloading asset: ${asset.name} (${asset.size} bytes)`);
const assetResponse = await fetch(asset.browser_download_url, {
headers: {
'Accept': 'application/octet-stream',
'Authorization': `token ${decryptedConfig.githubConfig.token}`,
},
});
if (!assetResponse.ok) {
console.error(`[Releases] Failed to download asset ${asset.name}: ${assetResponse.statusText}`);
continue;
}
const assetData = await assetResponse.arrayBuffer();
// Upload the asset to Gitea release
const formData = new FormData();
formData.append('attachment', new Blob([assetData]), asset.name);
const uploadResponse = await fetch(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${createReleaseResponse.data.id}/assets?name=${encodeURIComponent(asset.name)}`,
{
method: 'POST',
headers: {
'Authorization': `token ${decryptedConfig.giteaConfig.token}`,
},
body: formData,
}
);
if (uploadResponse.ok) {
console.log(`[Releases] Successfully uploaded asset: ${asset.name}`);
} else {
const errorText = await uploadResponse.text();
console.error(`[Releases] Failed to upload asset ${asset.name}: ${errorText}`);
}
} catch (assetError) {
console.error(`[Releases] Error processing asset ${asset.name}: ${assetError instanceof Error ? assetError.message : String(assetError)}`);
}
}
const assetResult = await reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId: createReleaseResponse.data.id,
githubAssets: release.assets,
tagName: release.tag_name,
});
totalAssetsUploaded += assetResult.uploaded;
totalAssetsFailed += assetResult.failed;
}
mirroredCount++;
const noteInfo = originalReleaseNote ? ` with ${originalReleaseNote.length} character changelog` : " without changelog";
console.log(`[Releases] Successfully mirrored release: ${release.tag_name}${noteInfo}`);
// Add delay to ensure proper timestamp ordering in Gitea
// Gitea sorts releases by created_unix DESC, and all releases created in quick succession
// will have nearly identical timestamps. The 1-second delay ensures proper chronological order.
console.log(`[Releases] Waiting 1 second to ensure proper timestamp ordering in Gitea...`);
await new Promise(resolve => setTimeout(resolve, 1000));
} catch (error) {
console.error(`[Releases] Failed to mirror release ${release.tag_name}: ${error instanceof Error ? error.message : String(error)}`);
}
}
console.log(`✅ Mirrored/Updated ${mirroredCount} releases to Gitea (${skippedCount} already up-to-date)`);
console.log(
`✅ Mirrored/Updated ${mirroredCount} releases to Gitea (${skippedCount} already up-to-date, ${skippedMissingTagCount} skipped: tag not synced yet); assets uploaded: ${totalAssetsUploaded}, failed: ${totalAssetsFailed}`
);
if (skippedMissingTagCount > 0) {
console.warn(
`[Releases] ${skippedMissingTagCount} release(s) skipped because their git tag is not in Gitea yet for ${repository.fullName} — these will be created automatically once the git mirror finishes syncing the tags`
);
}
if (totalAssetsFailed > 0) {
console.error(
`[Releases] ⚠️ ${totalAssetsFailed} release asset(s) failed to mirror for ${repository.fullName} — they will be retried on the next sync`
);
}
// Enforce release retention limit by removing the oldest excess releases from Gitea
try {
+164 -9
View File
@@ -1,15 +1,23 @@
import { describe, expect, test, mock } from "bun:test";
import { getGithubRepositories } from "@/lib/github";
function makeRepo() {
function makeRepo(overrides: Partial<{
name: string;
full_name: string;
ownerLogin: string;
ownerType: string;
fork: boolean;
}> = {}) {
const ownerLogin = overrides.ownerLogin ?? "octo";
const ownerType = overrides.ownerType ?? "User";
return {
name: "demo",
full_name: "octo/demo",
html_url: "https://github.com/octo/demo",
clone_url: "https://github.com/octo/demo.git",
owner: { login: "octo", type: "User" },
name: overrides.name ?? "demo",
full_name: overrides.full_name ?? `${ownerLogin}/${overrides.name ?? "demo"}`,
html_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}`,
clone_url: `https://github.com/${ownerLogin}/${overrides.name ?? "demo"}.git`,
owner: { login: ownerLogin, type: ownerType },
private: false,
fork: false,
fork: overrides.fork ?? false,
has_issues: true,
archived: false,
size: 1,
@@ -23,11 +31,11 @@ function makeRepo() {
};
}
function makeOctokit() {
function makeOctokit(reposToReturn?: ReturnType<typeof makeRepo>[]) {
let captured: Record<string, unknown> | null = null;
const paginate = mock(async (_method: unknown, options?: Record<string, unknown>) => {
captured = options ?? null;
return [makeRepo()];
return reposToReturn ?? [makeRepo()];
});
return {
octokit: {
@@ -98,3 +106,150 @@ describe("getGithubRepositories - affiliation", () => {
}
});
});
describe("getGithubRepositories - skipPersonalRepos", () => {
const personalRepo = makeRepo({ name: "my-lib", ownerLogin: "octo", ownerType: "User" });
const orgRepo = makeRepo({ name: "org-lib", ownerLogin: "my-org", ownerType: "Organization" });
const otherUserRepo = makeRepo({ name: "collab-lib", ownerLogin: "other-user", ownerType: "User" });
test("default false — keeps all repos including personal", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: false } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — drops repos owned by authenticated user", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — keeps repos owned by other users (collaborator repos)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo, otherUserRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", skipPersonalRepos: true } as any },
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
expect(repos.map((r) => r.name)).toContain("collab-lib");
});
test("skipPersonalRepos=true with no owner configured — keeps all repos (safe fallback)", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "", skipPersonalRepos: true } as any },
});
// Empty owner means we can't identify the user, so nothing should be dropped
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
test("skipPersonalRepos=true — unset (undefined) behaves like false", async () => {
const { octokit } = makeOctokit([personalRepo, orgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo" } as any },
});
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("org-lib");
});
});
describe("getGithubRepositories - includeOrganizations allowlist", () => {
const personalRepo = makeRepo({ name: "my-lib", ownerLogin: "octo", ownerType: "User" });
const wantedOrgRepo = makeRepo({ name: "wanted", ownerLogin: "wanted-org", ownerType: "Organization" });
const otherOrgRepo = makeRepo({ name: "noise", ownerLogin: "noise-org", ownerType: "Organization" });
const collabRepo = makeRepo({ name: "collab", ownerLogin: "other-user", ownerType: "User" });
test("empty allowlist — keeps repos from all orgs (backward compat)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: [] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
test("non-empty allowlist — keeps only listed orgs, drops other orgs", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("allowlist match is case-insensitive", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["Wanted-Org"] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("allowlist never restricts personal or collaborator repos", async () => {
const { octokit } = makeOctokit([personalRepo, collabRepo, wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
});
// User-owned and collaborator repos pass through regardless of the allowlist
expect(repos.map((r) => r.name)).toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("collab");
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("composes with skipPersonalRepos — drops personal, keeps only listed org", async () => {
const { octokit } = makeOctokit([personalRepo, wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: {
githubConfig: {
owner: "octo",
skipPersonalRepos: true,
includeOrganizations: ["wanted-org"],
} as any,
},
});
expect(repos.map((r) => r.name)).not.toContain("my-lib");
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).not.toContain("noise");
});
test("blank/whitespace entries are ignored (treated as empty allowlist)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: [" ", ""] } as any },
});
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
test("includeAllOrgsOverride bypasses allowlist (cleanup safety)", async () => {
const { octokit } = makeOctokit([wantedOrgRepo, otherOrgRepo]);
const repos = await getGithubRepositories({
octokit,
config: { githubConfig: { owner: "octo", includeOrganizations: ["wanted-org"] } as any },
includeAllOrgsOverride: true,
});
// Override returns all org repos so cleanup never false-orphans excluded orgs
expect(repos.map((r) => r.name)).toContain("wanted");
expect(repos.map((r) => r.name)).toContain("noise");
});
});
+40 -2
View File
@@ -236,6 +236,7 @@ export async function getGithubRepositories({
octokit,
config,
includeCollaboratorReposOverride,
includeAllOrgsOverride,
}: {
octokit: Octokit;
config: Partial<Config>;
@@ -243,6 +244,10 @@ export async function getGithubRepositories({
// cleanup service so we never mark a collab repo as orphaned just because
// the import filter is currently off.
includeCollaboratorReposOverride?: boolean;
// Bypass the includeOrganizations allowlist so all org repos are returned.
// Used by the cleanup service so a previously-mirrored org repo isn't flagged
// as orphaned just because the user narrowed the allowlist.
includeAllOrgsOverride?: boolean;
}): Promise<GitRepo[]> {
try {
const includeCollab =
@@ -263,10 +268,37 @@ export async function getGithubRepositories({
);
const skipForks = config.githubConfig?.skipForks ?? false;
const skipPersonalRepos = config.githubConfig?.skipPersonalRepos ?? false;
// The authenticated user's login — used to identify personally-owned repos
const authenticatedUserLogin = config.githubConfig?.owner ?? "";
// Opt-in organization allowlist. When non-empty, only repos owned by the
// listed organizations are imported; org repos from any other org the user
// happens to be a member of are dropped. Empty = import all org repos
// (backward-compatible default). Owned/collaborator repos are unaffected.
const includeOrgs = includeAllOrgsOverride
? []
: config.githubConfig?.includeOrganizations ?? [];
const allowedOrgs = new Set(
includeOrgs.map((org) => org.trim().toLowerCase()).filter(Boolean),
);
const filteredRepos = repos.filter((repo) => {
const isForkAllowed = !skipForks || !repo.fork;
return isForkAllowed;
// When skipPersonalRepos is true, drop repos owned by the authenticated user
// (owner.type === "User" and owner.login matches the configured GitHub username).
// Org repos have owner.type === "Organization" so they are always kept.
const isPersonalRepo =
skipPersonalRepos &&
authenticatedUserLogin.length > 0 &&
repo.owner.login === authenticatedUserLogin &&
repo.owner.type === "User";
// When an allowlist is configured, only keep org repos whose owning org
// is listed. Non-org repos (owned/collaborator) are never restricted here.
const isOrgAllowed =
allowedOrgs.size === 0 ||
repo.owner.type !== "Organization" ||
allowedOrgs.has(repo.owner.login.toLowerCase());
return isForkAllowed && !isPersonalRepo && isOrgAllowed;
});
return filteredRepos.map((repo) => ({
@@ -667,9 +699,11 @@ export async function getGithubStarredListNames({
export async function getGithubOrganizations({
octokit,
config,
skipOrgNames,
}: {
octokit: Octokit;
config: Partial<Config>;
skipOrgNames?: Set<string>;
}): Promise<{ organizations: GitOrg[]; failedOrgs: { name: string; avatarUrl: string; reason: string }[] }> {
try {
const { data: orgs } = await octokit.orgs.listForAuthenticatedUser({
@@ -682,7 +716,7 @@ export async function getGithubOrganizations({
? excludedOrgsEnv.split(",").map((org) => org.trim().toLowerCase())
: [];
// Filter out excluded organizations
// Filter out excluded and user-ignored organizations
const filteredOrgs = orgs.filter((org) => {
if (excludedOrgs.includes(org.login.toLowerCase())) {
console.log(
@@ -690,6 +724,10 @@ export async function getGithubOrganizations({
);
return false;
}
if (skipOrgNames?.has(org.login.toLowerCase())) {
console.log(`Skipping organization ${org.login} - ignored by user`);
return false;
}
return true;
});
+11 -5
View File
@@ -33,12 +33,18 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
let githubApiAccessible = true;
try {
// Fetch GitHub data. Always include collaborator repos here regardless
// of the user's import filter, otherwise repos previously mirrored as a
// collaborator would be flagged as orphaned and archived/deleted as soon
// as the user disables the filter.
// Fetch GitHub data. Always include collaborator repos and bypass the
// organization allowlist here regardless of the user's import filters,
// otherwise repos previously mirrored as a collaborator or from an org the
// user later removed from the allowlist would be flagged as orphaned and
// archived/deleted as soon as the user narrows those filters.
const [basicAndForkedRepos, starredRepos] = await Promise.all([
getGithubRepositories({ octokit, config, includeCollaboratorReposOverride: true }),
getGithubRepositories({
octokit,
config,
includeCollaboratorReposOverride: true,
includeAllOrgsOverride: true,
}),
config.githubConfig?.includeStarred
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
+22
View File
@@ -103,6 +103,28 @@ describe("Scheduler Service - Ignored Repository Handling", () => {
]);
});
test("auto-start gate: enabled=true → should start, enabled=false → should not start even with mirrorInterval", () => {
// Mirror the gate logic from checkAutoStartConfiguration / performInitialAutoStart.
// The enabled flag is the single authoritative signal; a configured
// mirrorInterval is a timing detail and must not bypass a disabled toggle.
const shouldAutoStart = (scheduleConfig?: { enabled?: boolean }) =>
scheduleConfig?.enabled === true;
expect(shouldAutoStart({ enabled: true })).toBe(true);
expect(shouldAutoStart({ enabled: false })).toBe(false);
expect(shouldAutoStart({})).toBe(false);
expect(shouldAutoStart(undefined)).toBe(false);
// Simulating: user disabled scheduling but has a mirrorInterval configured.
// The old code checked `scheduleEnabled || hasMirrorInterval`; the fix
// ensures only the enabled flag is checked.
const configWithIntervalButDisabled = {
scheduleConfig: { enabled: false },
giteaConfig: { mirrorInterval: "8h" },
};
expect(shouldAutoStart(configWithIntervalButDisabled.scheduleConfig)).toBe(false);
});
test("should validate all repository status enum values", () => {
const validStatuses = [
"imported",
+36 -10
View File
@@ -266,6 +266,27 @@ async function runScheduledSync(config: any): Promise<void> {
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
// A `failed` repo whose recorded location still resolves to a
// live same-source mirror (e.g. migrate succeeded but metadata
// failed) must be SYNCED, not re-created — otherwise the
// re-create loop spawns suffixed duplicates (#315). The create
// path also reuses now, but routing to sync here avoids a
// wasted migrate attempt and keeps recovery cheap.
if (repo.status === 'failed' && repository.mirroredLocation) {
const { findExistingMirror } = await import('@/lib/utils/mirror-source-match');
const existing = await findExistingMirror({
repository,
config,
candidateOwner: repository.mirroredLocation.split('/')[0] || '',
candidateName: repository.name,
});
if (existing) {
await syncGiteaRepo({ config, repository });
console.log(`[Scheduler] Re-synced failed repository with live mirror: ${repo.fullName}`);
return;
}
}
await mirrorGithubRepoToGitea({ octokit, repository, config });
console.log(`[Scheduler] Auto-mirrored repository: ${repo.fullName}`);
} catch (error) {
@@ -431,13 +452,16 @@ async function checkAutoStartConfiguration(): Promise<boolean> {
.where(eq(configs.isActive, true));
for (const config of activeConfigs) {
// Check if scheduling is enabled via environment
// Check if scheduling is enabled.
// Note: env-config-loader already sets scheduleConfig.enabled=true when
// GITEA_MIRROR_INTERVAL is set at startup, so the enabled flag is the
// single authoritative gate here. Checking hasMirrorInterval directly
// would allow a configured interval to trigger auto-start even after the
// user explicitly disabled scheduling via the UI.
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// If either SCHEDULE_ENABLED=true or GITEA_MIRROR_INTERVAL is set, we should auto-start
if (scheduleEnabled || hasMirrorInterval) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled}, hasMirrorInterval=${hasMirrorInterval})`);
if (scheduleEnabled) {
console.log(`[Scheduler] Auto-start conditions met for user ${config.userId} (scheduleEnabled=${scheduleEnabled})`);
return true;
}
}
@@ -472,10 +496,12 @@ async function performInitialAutoStart(): Promise<void> {
}
const scheduleEnabled = config.scheduleConfig?.enabled === true;
const hasMirrorInterval = !!config.giteaConfig?.mirrorInterval;
// Only process configs that have scheduling or mirror interval configured
if (!scheduleEnabled && !hasMirrorInterval) {
// Only process configs where scheduling is explicitly enabled.
// env-config-loader already sets enabled=true when GITEA_MIRROR_INTERVAL
// is present, so this single check covers both the UI toggle and the
// env-var boot path without letting a bare interval override a disabled toggle.
if (!scheduleEnabled) {
continue;
}
-303
View File
@@ -1,303 +0,0 @@
/**
* Enhanced handler for starred repositories with improved error handling
*/
import type { Config, Repository } from "./db/schema";
import { Octokit } from "@octokit/rest";
import { processWithRetry } from "./utils/concurrency";
import {
getOrCreateGiteaOrgEnhanced,
getGiteaRepoInfo,
handleExistingNonMirrorRepo,
createOrganizationsSequentially
} from "./gitea-enhanced";
import { mirrorGithubRepoToGitea } from "./gitea";
import { getMirrorStrategyConfig } from "./utils/mirror-strategies";
import { createMirrorJob } from "./helpers";
/**
* Process starred repositories with enhanced error handling
*/
export async function processStarredRepositories({
config,
repositories,
octokit,
}: {
config: Config;
repositories: Repository[];
octokit: Octokit;
}): Promise<void> {
if (!config.userId) {
throw new Error("User ID is required");
}
const strategyConfig = getMirrorStrategyConfig();
console.log(`Processing ${repositories.length} starred repositories`);
console.log(`Using strategy config:`, strategyConfig);
// Step 1: Pre-create organizations to avoid race conditions
if (strategyConfig.sequentialOrgCreation) {
await preCreateOrganizations({ config, repositories });
}
// Step 2: Process repositories with enhanced error handling
await processWithRetry(
repositories,
async (repository) => {
try {
await processStarredRepository({
config,
repository,
octokit,
strategyConfig,
});
return repository;
} catch (error) {
console.error(`Failed to process starred repository ${repository.name}:`, error);
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 2,
retryDelay: 2000,
onProgress: (completed, total, result) => {
const percentComplete = Math.round((completed / total) * 100);
if (result) {
console.log(
`Processed starred repository "${result.name}" (${completed}/${total}, ${percentComplete}%)`
);
}
},
onRetry: (repo, error, attempt) => {
console.log(
`Retrying starred repository ${repo.name} (attempt ${attempt}): ${error.message}`
);
},
}
);
}
/**
* Pre-create all required organizations sequentially
*/
async function preCreateOrganizations({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
// Get unique organization names
const orgNames = new Set<string>();
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
if (starredReposMode === "preserve-owner") {
for (const repo of repositories) {
orgNames.add(repo.organization || repo.owner);
}
} else if (config.githubConfig?.starredReposOrg) {
orgNames.add(config.githubConfig.starredReposOrg);
} else {
orgNames.add("starred");
}
// Add any other organizations based on mirror strategy
for (const repo of repositories) {
if (repo.destinationOrg) {
orgNames.add(repo.destinationOrg);
}
}
console.log(`Pre-creating ${orgNames.size} organizations sequentially`);
// Create organizations sequentially
await createOrganizationsSequentially({
config,
orgNames: Array.from(orgNames),
});
}
/**
* Process a single starred repository with enhanced error handling
*/
async function processStarredRepository({
config,
repository,
octokit,
strategyConfig,
}: {
config: Config;
repository: Repository;
octokit: Octokit;
strategyConfig: ReturnType<typeof getMirrorStrategyConfig>;
}): Promise<void> {
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
// Check if repository exists in Gitea
const existingRepo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (existingRepo) {
if (existingRepo.mirror) {
console.log(`Starred repository ${repository.name} already exists as a mirror`);
// Update database status
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
} else {
// Repository exists but is not a mirror
console.warn(`Starred repository ${repository.name} exists but is not a mirror`);
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo: existingRepo,
strategy: strategyConfig.nonMirrorStrategy,
});
// If we deleted it, continue to create the mirror
if (strategyConfig.nonMirrorStrategy !== "delete") {
return; // Skip if we're not deleting
}
}
}
// Create the mirror
try {
await mirrorGithubRepoToGitea({
octokit,
repository,
config,
});
} catch (error) {
// Enhanced error handling for specific scenarios
if (error instanceof Error) {
const errorMessage = error.message.toLowerCase();
if (errorMessage.includes("already exists")) {
// Handle race condition where repo was created by another process
console.log(`Repository ${repository.name} was created by another process`);
// Check if it's a mirror now
const recheck = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (recheck && recheck.mirror) {
// It's now a mirror, update database
const { db, repositories: reposTable } = await import("./db");
const { eq } = await import("drizzle-orm");
const { repoStatusEnum } = await import("@/types/Repository");
await db
.update(reposTable)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${starredOrg}/${repository.name}`,
})
.where(eq(reposTable.id, repository.id!));
return;
}
}
}
throw error;
}
}
/**
* Sync all starred repositories
*/
export async function syncStarredRepositories({
config,
repositories,
}: {
config: Config;
repositories: Repository[];
}): Promise<void> {
const strategyConfig = getMirrorStrategyConfig();
console.log(`Syncing ${repositories.length} starred repositories`);
await processWithRetry(
repositories,
async (repository) => {
try {
// Import syncGiteaRepo
const { syncGiteaRepo } = await import("./gitea");
await syncGiteaRepo({
config,
repository,
});
return repository;
} catch (error) {
if (error instanceof Error && error.message.includes("not a mirror")) {
console.warn(`Repository ${repository.name} is not a mirror, handling...`);
const starredReposMode = config.githubConfig?.starredReposMode || "dedicated-org";
const starredOrg =
starredReposMode === "preserve-owner"
? repository.organization || repository.owner
: config.githubConfig?.starredReposOrg || "starred";
const repoInfo = await getGiteaRepoInfo({
config,
owner: starredOrg,
repoName: repository.name,
});
if (repoInfo) {
await handleExistingNonMirrorRepo({
config,
repository,
repoInfo,
strategy: strategyConfig.nonMirrorStrategy,
});
}
}
throw error;
}
},
{
concurrencyLimit: strategyConfig.repoBatchSize,
maxRetries: 1,
retryDelay: 1000,
onProgress: (completed, total) => {
const percentComplete = Math.round((completed / total) * 100);
console.log(`Sync progress: ${completed}/${total} (${percentComplete}%)`);
},
}
);
}
+67
View File
@@ -124,3 +124,70 @@ test("githubConfigSchema parses includeCollaboratorRepos with true default", ()
});
expect(parsed.includeCollaboratorRepos).toBe(true);
});
test("skipPersonalRepos defaults to false in githubConfigSchema", () => {
const parsed = githubConfigSchema.parse({
owner: "octo",
type: "personal",
token: "",
});
expect(parsed.skipPersonalRepos).toBe(false);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when true", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: true };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(true);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(true);
});
test("skipPersonalRepos round-trips UI -> DB -> UI when false", () => {
const ui = buildMinimalUiConfigs();
const advancedWithSkip: AdvancedOptions = { ...ui.advancedOptions, skipPersonalRepos: false };
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, advancedWithSkip);
expect(db.githubConfig.skipPersonalRepos).toBe(false);
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.advancedOptions.skipPersonalRepos).toBe(false);
});
test("DB row missing skipPersonalRepos defaults to false on read", () => {
const ui = mapDbToUiConfig({ githubConfig: { owner: "octo", token: "" } });
expect(ui.advancedOptions.skipPersonalRepos).toBe(false);
});
// Regression for #326: the Name Collision Strategy dropdown didn't persist
// because starredDuplicateStrategy was missing from both mapper directions.
test("starredDuplicateStrategy round-trips UI -> DB -> UI when set to prefix", () => {
const ui = buildMinimalUiConfigs();
ui.githubConfig.starredDuplicateStrategy = "prefix";
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("prefix");
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.githubConfig.starredDuplicateStrategy).toBe("prefix");
});
test("starredDuplicateStrategy round-trips UI -> DB -> UI when set to suffix", () => {
const ui = buildMinimalUiConfigs();
ui.githubConfig.starredDuplicateStrategy = "suffix";
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("suffix");
const roundTripped = mapDbToUiConfig({ githubConfig: db.githubConfig, giteaConfig: db.giteaConfig });
expect(roundTripped.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
test("starredDuplicateStrategy defaults to suffix on save when unset", () => {
const ui = buildMinimalUiConfigs();
const db = mapUiToDbConfig(ui.githubConfig, ui.giteaConfig, ui.mirrorOptions, ui.advancedOptions);
expect(db.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
test("DB row missing starredDuplicateStrategy defaults to suffix on read", () => {
const ui = mapDbToUiConfig({ githubConfig: { owner: "octo", token: "" } });
expect(ui.githubConfig.starredDuplicateStrategy).toBe("suffix");
});
+25 -2
View File
@@ -31,6 +31,24 @@ function normalizeStarredLists(lists: string[] | undefined): string[] {
return [...deduped];
}
// Trim, drop blanks, and de-duplicate case-insensitively while preserving the
// first-seen casing of each organization name.
function normalizeOrgList(orgs: string[] | undefined): string[] {
if (!Array.isArray(orgs)) return [];
const seen = new Set<string>();
const result: string[] = [];
for (const org of orgs) {
if (typeof org !== "string") continue;
const trimmed = org.trim();
if (!trimmed) continue;
const key = trimmed.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
result.push(trimmed);
}
return result;
}
/**
* Maps UI config structure to database schema structure
*/
@@ -56,13 +74,14 @@ export function mapUiToDbConfig(
includeArchived: false, // Not in UI yet, default to false
includePublic: true, // Not in UI yet, default to true
// Organization related fields
includeOrganizations: [], // Not in UI yet
// Organization related fields — opt-in allowlist (empty = all org repos)
includeOrganizations: normalizeOrgList(githubConfig.includeOrganizations),
// Starred repos organization
starredReposOrg: giteaConfig.starredReposOrg,
starredReposMode: giteaConfig.starredReposMode || "dedicated-org",
starredLists: normalizeStarredLists(githubConfig.starredLists),
starredDuplicateStrategy: githubConfig.starredDuplicateStrategy ?? "suffix",
// Mirror strategy
mirrorStrategy: giteaConfig.mirrorStrategy || "preserve",
@@ -71,6 +90,7 @@ export function mapUiToDbConfig(
// Advanced options
starredCodeOnly: advancedOptions.starredCodeOnly,
autoMirrorStarred: advancedOptions.autoMirrorStarred ?? false,
skipPersonalRepos: advancedOptions.skipPersonalRepos ?? false,
};
// Map Gitea config to match database schema
@@ -144,8 +164,10 @@ export function mapDbToUiConfig(dbConfig: any): {
token: dbConfig.githubConfig?.token || "",
privateRepositories: dbConfig.githubConfig?.includePrivate || false, // Map includePrivate to privateRepositories
includeCollaboratorRepos: dbConfig.githubConfig?.includeCollaboratorRepos ?? true,
includeOrganizations: normalizeOrgList(dbConfig.githubConfig?.includeOrganizations),
mirrorStarred: dbConfig.githubConfig?.includeStarred || false, // Map includeStarred to mirrorStarred
starredLists: normalizeStarredLists(dbConfig.githubConfig?.starredLists),
starredDuplicateStrategy: dbConfig.githubConfig?.starredDuplicateStrategy ?? "suffix",
};
// Map from database Gitea config to UI fields
@@ -194,6 +216,7 @@ export function mapDbToUiConfig(dbConfig: any): {
// Support both old (skipStarredIssues) and new (starredCodeOnly) field names for backward compatibility
starredCodeOnly: dbConfig.githubConfig?.starredCodeOnly ?? (dbConfig.githubConfig as any)?.skipStarredIssues ?? false,
autoMirrorStarred: dbConfig.githubConfig?.autoMirrorStarred ?? false,
skipPersonalRepos: dbConfig.githubConfig?.skipPersonalRepos ?? false,
};
return {
+420
View File
@@ -0,0 +1,420 @@
import { describe, test, expect } from "bun:test";
import {
normalizeCloneUrl,
cloneUrlsMatch,
isMirrorOfSource,
classifyCandidateName,
findExistingMirror,
} from "./mirror-source-match";
import type { Repository } from "@/lib/db/schema";
import type { Config } from "@/types/config";
// Minimal Repository factory for tests. Only the fields read by the helper
// matter (cloneUrl, mirroredLocation, fullName, name).
function makeRepo(overrides: Partial<Repository> = {}): Repository {
return {
id: "repo-1",
userId: "user-1",
configId: "config-1",
name: "Update",
fullName: "NostalgiaForInfinity/Update",
url: "https://github.com/NostalgiaForInfinity/Update",
cloneUrl: "https://github.com/NostalgiaForInfinity/Update.git",
owner: "NostalgiaForInfinity",
organization: undefined,
mirroredLocation: "",
isPrivate: false,
isForked: false,
forkedFrom: undefined,
hasIssues: false,
isStarred: true,
isArchived: false,
size: 0,
hasLFS: false,
hasSubmodules: false,
language: undefined,
description: undefined,
defaultBranch: "main",
visibility: "public",
status: "imported",
lastMirrored: undefined,
errorMessage: undefined,
createdAt: new Date(),
updatedAt: new Date(),
...overrides,
} as unknown as Repository;
}
const config: Partial<Config> = {
userId: "user-1",
giteaConfig: { url: "https://gitea.example.com", token: "t" } as any,
};
describe("normalizeCloneUrl", () => {
test("strips trailing .git", () => {
expect(normalizeCloneUrl("https://github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips embedded credentials", () => {
expect(normalizeCloneUrl("https://x-access-token:ghp_secret@github.com/a/b.git")).toBe(
"https://github.com/a/b"
);
});
test("strips trailing slash", () => {
expect(normalizeCloneUrl("https://github.com/a/b/")).toBe(
"https://github.com/a/b"
);
});
test("lowercases host (and value)", () => {
expect(normalizeCloneUrl("https://GitHub.com/a/b")).toBe(
"https://github.com/a/b"
);
});
test("returns empty string for blank/invalid input", () => {
expect(normalizeCloneUrl("")).toBe("");
expect(normalizeCloneUrl(null)).toBe("");
expect(normalizeCloneUrl(undefined)).toBe("");
});
test("handles scp-style git URLs via fallback", () => {
expect(normalizeCloneUrl("git@github.com:a/b.git")).toBe("git@github.com:a/b");
});
});
describe("cloneUrlsMatch", () => {
test("https vs token-embedded URL match", () => {
expect(
cloneUrlsMatch(
"https://github.com/a/b.git",
"https://x-access-token:tok@github.com/a/b.git"
)
).toBe(true);
});
test(".git suffix and trailing slash differences match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/a/b.git/")
).toBe(true);
});
test("host case-insensitive match", () => {
expect(
cloneUrlsMatch("https://GITHUB.com/a/b", "https://github.com/a/b")
).toBe(true);
});
test("different repos do not match", () => {
expect(
cloneUrlsMatch("https://github.com/a/b", "https://github.com/c/d")
).toBe(false);
});
test("empty/unknown URL never matches", () => {
expect(cloneUrlsMatch("", "https://github.com/a/b")).toBe(false);
expect(cloneUrlsMatch("https://github.com/a/b", undefined)).toBe(false);
});
});
describe("isMirrorOfSource", () => {
test("true when mirror with matching original_url", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b.git"
)
).toBe(true);
});
test("false when not a mirror", () => {
expect(
isMirrorOfSource(
{ mirror: false, original_url: "https://github.com/a/b" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url is for a different source (phantom fork)", () => {
expect(
isMirrorOfSource(
{ mirror: true, original_url: "https://github.com/other/repo" } as any,
"https://github.com/a/b"
)
).toBe(false);
});
test("false when original_url missing (cannot confirm)", () => {
expect(
isMirrorOfSource({ mirror: true } as any, "https://github.com/a/b")
).toBe(false);
});
test("false for null repoInfo", () => {
expect(isMirrorOfSource(null, "https://github.com/a/b")).toBe(false);
});
});
describe("findExistingMirror", () => {
test("reuses existing same-source mirror at base candidate name (#315)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
expect(match!.repoName).toBe("Update");
});
test("reuses via mirroredLocation even when base name differs (strategy change, #309)", async () => {
// Strategy changed; current candidate name would be "Update" under "starred",
// but the historical mirror lives at "myorg/Update-NostalgiaForInfinity".
const repo = makeRepo({
mirroredLocation: "myorg/Update-NostalgiaForInfinity",
});
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "myorg" && repoName === "Update-NostalgiaForInfinity") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("myorg");
expect(match!.repoName).toBe("Update-NostalgiaForInfinity");
});
test("returns null on genuine different-source collision (regression guard #95/#236)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
// Same name, but it mirrors a DIFFERENT source.
return {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("returns null for phantom fork (non-mirror at the name)", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return { mirror: false, original_url: "" } as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("falls back to fresh creation when mirroredLocation is stale (Gitea repo deleted)", async () => {
const repo = makeRepo({ mirroredLocation: "starred/Update" });
// Both the recorded location and the base candidate are gone.
const getRepoInfo = async () => null;
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).toBeNull();
});
test("matches mirror even when original_url is token-embedded / .git-suffixed", async () => {
const repo = makeRepo();
const getRepoInfo = async ({ owner, repoName }: any) => {
if (owner === "starred" && repoName === "Update") {
return {
mirror: true,
original_url:
"https://x-access-token:tok@github.com/NostalgiaForInfinity/Update.git",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
});
test("skips a candidate whose lookup throws and still resolves a later candidate", async () => {
const repo = makeRepo({ mirroredLocation: "myorg/Update" });
const getRepoInfo = async ({ owner }: any) => {
if (owner === "myorg") {
throw new Error("network blip");
}
if (owner === "starred") {
return {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any;
}
return null;
};
const match = await findExistingMirror({
repository: repo,
config,
candidateOwner: "starred",
candidateName: "Update",
getRepoInfo,
});
expect(match).not.toBeNull();
expect(match!.owner).toBe("starred");
});
});
describe("classifyCandidateName — suffix vs reuse decision (#315/#309)", () => {
const SOURCE = "https://github.com/NostalgiaForInfinity/Update.git";
test("free name → available", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: false,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("available");
});
test("name occupied by OUR same-source mirror → reusable (no suffix, #315)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("reusable");
});
test("name occupied by a DIFFERENT source → taken (suffix, regression #95/#236)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/someoneelse/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("name occupied by a NON-mirror → taken (phantom-fork guard, #309)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: { mirror: false, original_url: "" } as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("our same-source mirror but DB-claimed by ANOTHER repo → taken (per-user separation)", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: true,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("free in Gitea but DB-claimed by another concurrent op → taken", () => {
expect(
classifyCandidateName({
existsInGitea: false,
claimedByOther: true,
repoInfo: null,
sourceCloneUrl: SOURCE,
})
).toBe("taken");
});
test("existing mirror but unknown source (no sourceCloneUrl) → taken", () => {
expect(
classifyCandidateName({
existsInGitea: true,
claimedByOther: false,
repoInfo: {
mirror: true,
original_url: "https://github.com/NostalgiaForInfinity/Update",
} as any,
sourceCloneUrl: undefined,
})
).toBe("taken");
});
});
+213
View File
@@ -0,0 +1,213 @@
import type { Config } from "@/types/config";
import type { Repository } from "@/lib/db/schema";
import type { GiteaRepoInfo } from "@/lib/gitea-enhanced";
/**
* Source-identity matching for mirror reuse.
*
* Starred (and other) repos were duplicating on every re-mirror because the
* existence check only asked "does a repo with this name exist?" never
* "is the existing repo a mirror of THIS same GitHub source?". This module
* answers the second question so callers can reuse an existing same-source
* mirror instead of generating a suffixed duplicate. See issues #315 / #309.
*/
/**
* Normalize a git clone URL for source-identity comparison.
* Strips embedded credentials, a trailing ".git", a trailing slash, and
* lowercases the host (hosts are case-insensitive; paths are not). Returns an
* empty string for blank/invalid input so callers can treat it as "unknown".
*/
export function normalizeCloneUrl(rawUrl?: string | null): string {
if (typeof rawUrl !== "string") return "";
let url = rawUrl.trim();
if (!url) return "";
try {
const parsed = new URL(url);
// Drop any embedded credentials (e.g. https://user:token@host/...).
parsed.username = "";
parsed.password = "";
const host = parsed.host.toLowerCase();
// Strip trailing slash(es) first so a ".git/" suffix still normalizes.
const path = parsed.pathname.replace(/\/+$/, "").replace(/\.git$/i, "");
return `${parsed.protocol}//${host}${path}`.toLowerCase();
} catch {
// Fall back to best-effort string normalization for non-standard URLs
// (e.g. scp-style git@host:owner/repo). Strip credentials before "@",
// drop ".git"/trailing slash, and lowercase the whole thing.
url = url.replace(/^([a-z]+:\/\/)[^@/]+@/i, "$1");
url = url.replace(/\/+$/, "").replace(/\.git$/i, "");
return url.toLowerCase();
}
}
/**
* Whether two clone URLs point at the same source repository, ignoring
* credentials, ".git" suffix, trailing slash, and host case.
*/
export function cloneUrlsMatch(a?: string | null, b?: string | null): boolean {
const normA = normalizeCloneUrl(a);
const normB = normalizeCloneUrl(b);
if (!normA || !normB) return false;
return normA === normB;
}
/**
* Whether an existing Gitea repo is a mirror of the given GitHub source.
* Uses Gitea's original_url (the recorded migration source) when present;
* if Gitea didn't expose original_url, we cannot positively confirm the
* source and return false (callers then treat the name as a genuine
* collision rather than risk mapping onto an unrelated repo #309).
*/
export function isMirrorOfSource(
repoInfo: GiteaRepoInfo | null,
sourceCloneUrl?: string | null
): boolean {
if (!repoInfo || !repoInfo.mirror) return false;
return cloneUrlsMatch(repoInfo.original_url, sourceCloneUrl);
}
export type CandidateNameClassification = "available" | "reusable" | "taken";
/**
* Classify a candidate mirror name for the suffix-vs-reuse decision in
* generateUniqueRepoName. Pure (all I/O is pre-resolved by the caller):
* - "available": free in Gitea and not DB-claimed by another repo use it
* - "reusable": occupied in Gitea by a mirror of THIS source, not DB-claimed
* by another repo reuse it (no suffix)
* - "taken": occupied by a different source / non-mirror, or DB-claimed by
* another repo must suffix
*
* A DB claim by a DIFFERENT repo always blocks reuse so two users mirroring the
* same source into a shared org stay separated.
*/
export function classifyCandidateName({
existsInGitea,
claimedByOther,
repoInfo,
sourceCloneUrl,
}: {
existsInGitea: boolean;
claimedByOther: boolean;
repoInfo: GiteaRepoInfo | null;
sourceCloneUrl?: string | null;
}): CandidateNameClassification {
if (existsInGitea) {
if (!claimedByOther && isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return "reusable";
}
return "taken";
}
// Not in Gitea, but possibly claimed in the DB by a concurrent operation.
if (claimedByOther) return "taken";
return "available";
}
export interface ExistingMirrorMatch {
owner: string;
repoName: string;
repoInfo: GiteaRepoInfo;
}
/**
* Resolve an existing same-source mirror for a repository, if one exists.
*
* Resolution order (backward compatible):
* 1. The recorded repository.mirroredLocation if it still resolves to a
* live mirror of THIS source, reuse it even when the base candidate name
* differs from the current naming strategy (handles strategy changes #309).
* 2. The provided candidate owner/name if that resolves to a live mirror of
* THIS source, reuse it (handles the self-collision that drove suffixing #315).
*
* Returns null when no live same-source mirror is found (caller should create
* a fresh mirror, generating a unique name if the candidate name is taken by a
* DIFFERENT source).
*/
export async function findExistingMirror({
repository,
config,
candidateOwner,
candidateName,
getRepoInfo,
}: {
repository: Repository;
config: Partial<Config>;
candidateOwner: string;
candidateName: string;
// Injectable for testing; defaults to the real Gitea lookup.
getRepoInfo?: (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => Promise<GiteaRepoInfo | null>;
}): Promise<ExistingMirrorMatch | null> {
const lookup =
getRepoInfo ??
(async (args: {
config: Partial<Config>;
owner: string;
repoName: string;
}) => {
const { getGiteaRepoInfo } = await import("@/lib/gitea-enhanced");
return getGiteaRepoInfo(args);
});
const sourceCloneUrl = repository.cloneUrl;
// Candidate locations to probe, in priority order. Dedupe so we don't issue
// the same HTTP lookup twice when mirroredLocation equals the candidate.
const candidates: Array<{ owner: string; repoName: string }> = [];
const seen = new Set<string>();
const pushCandidate = (owner?: string | null, repoName?: string | null) => {
const o = (owner || "").trim();
const r = (repoName || "").trim();
if (!o || !r) return;
const key = `${o}/${r}`.toLowerCase();
if (seen.has(key)) return;
seen.add(key);
candidates.push({ owner: o, repoName: r });
};
if (repository.mirroredLocation && repository.mirroredLocation.trim()) {
const slashIndex = repository.mirroredLocation.indexOf("/");
if (slashIndex > 0 && slashIndex < repository.mirroredLocation.length - 1) {
pushCandidate(
repository.mirroredLocation.slice(0, slashIndex),
repository.mirroredLocation.slice(slashIndex + 1)
);
}
}
pushCandidate(candidateOwner, candidateName);
for (const candidate of candidates) {
let repoInfo: GiteaRepoInfo | null;
try {
repoInfo = await lookup({
config,
owner: candidate.owner,
repoName: candidate.repoName,
});
} catch (error) {
// A failed lookup (network/auth) should not be mistaken for "no mirror";
// skip this candidate and let the caller fall back to its normal flow.
console.warn(
`[Mirror] Could not look up ${candidate.owner}/${candidate.repoName} while resolving existing mirror for ${repository.fullName}: ${
error instanceof Error ? error.message : String(error)
}`
);
continue;
}
if (isMirrorOfSource(repoInfo, sourceCloneUrl)) {
return {
owner: candidate.owner,
repoName: candidate.repoName,
repoInfo: repoInfo as GiteaRepoInfo,
};
}
}
return null;
}
+89
View File
@@ -0,0 +1,89 @@
import type { APIRoute } from "astro";
import { db, repositories } from "@/lib/db";
import { and, eq, inArray } from "drizzle-orm";
import { repoStatusEnum } from "@/types/Repository";
import { createMirrorJob } from "@/lib/helpers";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
/**
* POST /api/job/cancel-pending
*
* Sets this user's repositories that are waiting to be mirrored
* (status: "imported" or "failed") to "ignored", preventing the scheduler
* from picking them up. Repos with status "mirroring" or "syncing" are
* left alone because they have in-flight work that cannot be aborted here.
*
* Returns the count of affected repositories and logs one activity entry.
*/
export const POST: APIRoute = async ({ request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
// Statuses that represent queued-but-not-started work.
// "imported" → repo was discovered, never mirrored
// "failed" → last mirror attempt failed; scheduler will retry
const cancelableStatuses = ["imported", "failed"] as const;
// Fetch repos to cancel so we can count them and log meaningful details.
const toCancel = await db
.select({ id: repositories.id })
.from(repositories)
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.status, cancelableStatuses),
),
);
const cancelCount = toCancel.length;
if (cancelCount > 0) {
const ids = toCancel.map((r) => r.id);
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("ignored"),
updatedAt: new Date(),
errorMessage: "Cancelled by user — set to ignored via Stop Pending Mirrors.",
})
.where(
and(
eq(repositories.userId, userId),
inArray(repositories.id, ids),
),
);
}
// Log a single activity summarising the bulk action.
await createMirrorJob({
userId,
message: `Stopped pending mirrors: ${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} set to Ignored`,
details:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y" : "ies"} with status "imported" or "failed" have been set to "ignored". ` +
`They can be re-enabled from the Repositories page.`
: `No repositories in a pending state were found for this user.`,
status: cancelCount > 0 ? "ignored" : "skipped",
skipDuplicateEvent: false,
skipNotification: true,
});
return new Response(
JSON.stringify({
success: true,
message:
cancelCount > 0
? `${cancelCount} repositor${cancelCount === 1 ? "y has" : "ies have"} been set to Ignored.`
: "No repositories in a pending state were found.",
cancelledCount: cancelCount,
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
} catch (error) {
return createSecureErrorResponse(error, "cancel pending mirrors", 500);
}
};
+1 -51
View File
@@ -1,5 +1,5 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { db, repositories } from "@/lib/db";
import { eq, and } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
@@ -62,53 +62,3 @@ export const PATCH: APIRoute = async (context) => {
}
};
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const repoId = context.params.id;
if (!repoId) {
return new Response(JSON.stringify({ error: "Repository ID is required" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
const [existingRepo] = await db
.select()
.from(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)))
.limit(1);
if (!existingRepo) {
return new Response(
JSON.stringify({ error: "Repository not found" }),
{
status: 404,
headers: { "Content-Type": "application/json" },
}
);
}
await db
.delete(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)));
await db
.delete(mirrorJobs)
.where(and(eq(mirrorJobs.repositoryId, repoId), eq(mirrorJobs.userId, userId)));
return new Response(
JSON.stringify({ success: true }),
{
status: 200,
headers: { "Content-Type": "application/json" },
}
);
} catch (error) {
return createSecureErrorResponse(error, "Delete repository", 500);
}
};
+49
View File
@@ -0,0 +1,49 @@
import type { APIRoute } from "astro";
import { db, repositories, mirrorJobs } from "@/lib/db";
import { eq, and, inArray } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
export const DELETE: APIRoute = async (context) => {
try {
const { user, response } = await requireAuth(context);
if (response) return response;
const userId = user!.id;
const body = await context.request.json();
const { ids } = body;
if (!Array.isArray(ids) || ids.length === 0) {
return new Response(JSON.stringify({ error: "ids must be a non-empty array" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
// Verify all repos belong to this user before deleting
const owned = await db
.select({ id: repositories.id })
.from(repositories)
.where(and(inArray(repositories.id, ids), eq(repositories.userId, userId)));
const ownedIds = owned.map((r) => r.id);
if (ownedIds.length === 0) {
return new Response(JSON.stringify({ error: "No matching repositories found" }), {
status: 404,
headers: { "Content-Type": "application/json" },
});
}
await db.transaction(async (tx) => {
await tx.delete(mirrorJobs).where(and(inArray(mirrorJobs.repositoryId, ownedIds), eq(mirrorJobs.userId, userId)));
await tx.delete(repositories).where(and(inArray(repositories.id, ownedIds), eq(repositories.userId, userId)));
});
return new Response(
JSON.stringify({ success: true, deleted: ownedIds.length }),
{ status: 200, headers: { "Content-Type": "application/json" } }
);
} catch (error) {
return createSecureErrorResponse(error, "Bulk delete repositories", 500);
}
};
+12 -3
View File
@@ -49,13 +49,20 @@ export const POST: APIRoute = async ({ request, locals }) => {
const githubUsername = config.githubConfig?.owner || undefined;
const octokit = createGitHubClient(decryptedToken, userId, githubUsername);
// Load ignored orgs from the DB so we can skip them during import
const ignoredOrgRows = await db
.select({ normalizedName: organizations.normalizedName })
.from(organizations)
.where(and(eq(organizations.userId, userId), eq(organizations.status, "ignored")));
const ignoredOrgNames = new Set(ignoredOrgRows.map((o) => o.normalizedName));
// Fetch GitHub data in parallel
const [basicAndForkedRepos, starredRepos, orgResult] = await Promise.all([
getGithubRepositories({ octokit, config }),
config.githubConfig?.includeStarred
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
getGithubOrganizations({ octokit, config }),
getGithubOrganizations({ octokit, config, skipOrgNames: ignoredOrgNames }),
]);
const { organizations: gitOrgs, failedOrgs } = orgResult;
@@ -152,7 +159,9 @@ export const POST: APIRoute = async ({ request, locals }) => {
const existingOrgMap = new Map(existingOrgs.map((o) => [o.normalizedName, o.status]));
insertedRepos = newRepos.filter(
(r) => !existingRepoNames.has(r.normalizedFullName)
(r) =>
!existingRepoNames.has(r.normalizedFullName) &&
(!r.organization || !ignoredOrgNames.has(r.organization.toLowerCase()))
);
insertedOrgs = newOrgs.filter((o) => !existingOrgMap.has(o.normalizedName));
@@ -258,7 +267,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
newRepositories: insertedRepos.length,
newOrganizations: insertedOrgs.length,
skippedDisabledRepositories: allGithubRepos.length - mirrorableGithubRepos.length,
failedOrgs: failedOrgs.map((o) => o.name),
failedOrgs: failedOrgs.filter((o) => !ignoredOrgNames.has(o.name.toLowerCase())).map((o) => o.name),
recoveredOrgs: recoveredOrgCount,
},
});
+2
View File
@@ -62,6 +62,7 @@ export interface GitHubConfig {
token: string;
privateRepositories: boolean;
includeCollaboratorRepos?: boolean;
includeOrganizations?: string[];
mirrorStarred: boolean;
starredLists?: string[];
starredDuplicateStrategy?: DuplicateNameStrategy;
@@ -86,6 +87,7 @@ export interface AdvancedOptions {
skipForks: boolean;
starredCodeOnly: boolean;
autoMirrorStarred?: boolean;
skipPersonalRepos?: boolean;
}
export interface SaveConfigApiRequest {
+11 -11
View File
@@ -9,24 +9,24 @@
"astro": "astro"
},
"dependencies": {
"@astrojs/mdx": "^5.0.0",
"@astrojs/react": "^5.0.0",
"@radix-ui/react-slot": "^1.2.4",
"@astrojs/mdx": "^5.0.6",
"@astrojs/react": "^5.0.7",
"@radix-ui/react-slot": "^1.2.5",
"@splinetool/react-spline": "^4.1.0",
"@splinetool/runtime": "^1.12.69",
"@tailwindcss/vite": "^4.2.1",
"@splinetool/runtime": "^1.12.97",
"@tailwindcss/vite": "^4.3.1",
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.2.14",
"@types/react": "^19.2.17",
"@types/react-dom": "^19.2.3",
"astro": "^6.1.6",
"astro": "^6.4.6",
"canvas-confetti": "^1.9.4",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^0.577.0",
"react": "^19.2.4",
"react-dom": "^19.2.4",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.2.1"
"react": "^19.2.7",
"react-dom": "^19.2.7",
"tailwind-merge": "^3.6.0",
"tailwindcss": "^4.3.1"
},
"devDependencies": {
"tw-animate-css": "^1.4.0"
+741 -745
View File
File diff suppressed because it is too large Load Diff