Compare commits

...

8 Commits

Author SHA1 Message Date
Arunavo Ray 06bfb49e0e chore: bump version to 3.20.4 2026-07-02 15:46:20 +05:30
ARUNAVO RAY b5e0c58708 fix: stop false-positive orphan archiving and heal sync 405s on archived-* renamed mirrors (#331) (#336)
Three related fixes for the "repos keep getting archived and then fail to
sync with HTTP 405" report:

1. Orphan cleanup no longer archives on bulk-list absence alone.
   Repos added via the "+" Add Repository dialog (foreign owner, not
   starred) can never appear in the authenticated bulk fetches, so every
   cleanup cycle deterministically flagged them as orphaned and archived
   them. identifyOrphanedRepositories() now runs a targeted per-repo
   confirmation (starred check or repos.get) and only treats a clean 404
   as gone; any other outcome fails safe.

2. The archived-* rename is persisted. archiveGiteaRepo() now returns
   the actual post-rename name and the cleanup service records it in
   mirroredLocation, so the DB no longer points at a name that only
   301-redirects.

3. Sync self-heals repos renamed in Gitea/Forgejo. Requests to a
   renamed repo get a 301; fetch follows it, downgrading POST to GET,
   which lands on the POST-only mirror-sync endpoint as a 405. The sync
   candidate loop now adopts the canonical owner/name from the GET
   response body before POSTing, tries an archived-{name} fallback for
   archived repos (guarded by an original_url source match), and keeps
   archived repos archived: no mirror-interval PATCH, status stays
   'archived' per the documented Manual Sync contract.

Also hardens mirrorGitHubReleasesToGitea to derive GitHub coordinates
from fullName so Gitea-side names can never leak into GitHub API calls.

Verified end-to-end on Forgejo 15.0.3 (rootless): pre-fix reproduces the
exact 405; post-fix the stale-name sync succeeds (GET stale -> 301, GET
canonical -> 200, POST canonical mirror-sync -> 200), archived repos keep
interval "0s" with no PATCH issued, and non-archived renamed repos heal
and get the configured interval applied.
2026-07-02 15:46:00 +05:30
Arunavo Ray 74606f0a5f chore: bump version to 3.20.3 2026-07-01 08:13:15 +05:30
ARUNAVO RAY 187ecc5d60 fix: correctly mirror Gitea release titles and issue/PR labels (#334 + sibling) (#335)
* fix(releases): send Gitea release title as `name`, not `title` (#334)

Gitea/Forgejo expose the release title through the JSON field `name`
(the API Go struct is `Title string `+"`"+`json:"name"`+"`"+`). The release
create and update payloads sent `title:` instead, which Gitea silently
ignores, so every mirrored release landed with a blank title.

Verified live against Gitea 1.24.7: a POST/PATCH with `title` yields
`name: ""`; the same call with `name` sets the title correctly. The
update path also self-heals previously-mirrored releases whose names were
left blank, since the existing-vs-expected name comparison already drives
a PATCH.

Adds gita-release-name.test.ts, which drives the real
mirrorGitHubReleasesToGitea create/update paths with a mocked fetch and
asserts the payload carries `name` (and never `title`).

* fix(issues): reconcile labels on issue/PR update via the labels sub-resource (#334 sibling)

Gitea/Forgejo's `EditIssueOption` has no `labels` field (only
`CreateIssueOption` does), so a `labels` key in a `PATCH .../issues/{index}`
body is silently dropped — the same silent-ignore class as the release
`title` vs `name` bug. The issue and PR-as-issue update paths sent `labels`
in the PATCH body, so label changes never propagated onto already-mirrored
issues (and a deadlock-orphaned issue recovered via PATCH never got its
labels).

Fix: add `reconcileGiteaIssueLabels`, which replaces the label set via
`PUT .../issues/{index}/labels` (idempotent — adds new, removes deleted).
Call it on the two issue update paths and the two PR-issue update paths,
and drop the dead `labels` key from those PATCH bodies. Labels on freshly
created issues still come from CreateIssueOption on the POST.

Verified live against Gitea 1.24.7 (PATCH ignores labels; PUT applies them)
and end-to-end (a drifted mirrored issue reconciled from no-labels to its
GitHub label set). Adds gitea-issue-labels.test.ts driving the real
mirrorGitRepoIssuesToGitea update path; the test carries a self-contained
http-client mock so it is immune to another suite's global module mock.

* test: make #334 regression tests deterministic via pure payload builders

The prior tests drove the real mirror functions with a global `fetch` mock.
That is order/version-fragile: another suite installs a process-global
`mock.module("@/lib/http-client")`, and bun 1.3.13 (CI) runs test files
concurrently, so `globalThis.fetch` races across files and
`isRepoPresentInGitea` (raw fetch) intermittently sees the wrong mock —
green locally on bun 1.3.6, red in CI.

Extract the payload construction into pure, exported builders and assert on
those instead (the repo's existing `classify*` pattern): buildGiteaReleasePayload
(create+update send `name`, never `title`), buildGiteaIssueEditPayload (edit
body never carries `labels`), buildGiteaIssueLabelsPayload (labels sub-resource
body). Behavior is unchanged — the builders return the exact same objects the
call sites built inline — and the fixes remain verified live on Gitea 1.24.7.
2026-07-01 08:12:36 +05:30
Arunavo Ray 632bbd0d4a chore: bump version to 3.20.2 2026-06-24 17:19:10 +05:30
ARUNAVO RAY 0b65e40784 fix(releases): create releases only for tags present in Gitea; stop sending target (#331) (#333)
Release creation failed on some Gitea/Forgejo instances with
"HTTP 404: The target couldn't be found", so no release (and therefore no
assets) was ever created — re-syncing never recovered.

Root cause: the create payload always sent `target: target_commitish`
(e.g. "main"). When the release's git tag is not yet present in the Gitea
mirror — which happens when Gitea's own git mirror clone lags behind the
metadata sync — Gitea tries to *create* the tag from `target`; if that ref
can't be resolved it returns a generic 404 ("The target couldn't be
found"), and if it can, it would create a brand-new tag at the wrong commit.

Reproduced the reporter's exact stack (Forgejo 15 rootless + read_only +
cap_drop ALL + postgres, plus Gitea 1.20-1.26 and Forgejo 1.21-15): a
healthy repo always succeeds — the 404 only occurs when the tag is absent
at create time.

Fix:
- Before creating a release, verify the git tag already exists in Gitea.
  If it isn't synced yet, skip it (logged) and let a later sync create it
  once the mirror has the tag — never create a tag via `target`.
- Drop the `target` field from both the create and update payloads. For a
  mirror the tag is synced from upstream, so Gitea attaches the release to
  the existing tag; `target` is unnecessary and is what triggers the 404.
- Surface skipped-missing-tag releases in the summary log for diagnosability.

Verified end-to-end against the real mirror function on a Forgejo instance:
a release whose tag exists is created with its assets; a release whose tag
was removed is skipped cleanly (no 404, no bogus tag) and picked up once the
tag is present.
2026-06-24 17:18:46 +05:30
Arunavo Ray 4a8b4f6ff3 chore: bump version to 3.20.1 2026-06-23 23:07:16 +05:30
ARUNAVO RAY 1d9dfdeb70 fix(releases): mirror assets idempotently so missing assets self-heal (#331) (#332)
Release assets were uploaded only on the create path of
mirrorGitHubReleasesToGitea(). When a Gitea release already existed, the
update path PATCHed the changelog/title and `continue`d without ever
touching assets. So any release whose assets were not fully uploaded on
that single create-path run — first sync interrupted, a transient
download/upload failure, large multi-MB assets, etc. — stayed permanently
asset-less, and re-syncing always hit the update path and could never
recover it. Asset failures were also swallowed to console.error, so the
job still reported success (the "no errors in the logs" in #331).

Reproduced on a real Forgejo pull-mirror with shauninman/MinUI: a GitHub
release with two ~35-40MB binaries became a Gitea release with 0 assets
(just Forgejo's auto-generated source archive), and re-syncing left it at
0 while logging "Updating existing release".

Fix:
- Add reconcileReleaseAssets(), an idempotent reconciler run on BOTH the
  create and update paths. It compares Gitea's existing attachments to
  GitHub's by name+size: skips matches, uploads missing ones, replaces
  size-mismatched copies. Existing broken releases self-heal on next sync.
- Extract the pure decision into classifyAssetsForReconciliation() for
  unit testing (the absence of asset tests is why this slipped past #310).
- Surface asset upload counts in the summary and emit a visible warning
  when any fail, instead of silently swallowing them.
- Add 5 unit tests covering the broken state, partial backfill,
  idempotency, size-mismatch replacement, and the no-assets case.

Verified end-to-end against the real function on a Forgejo pull-mirror:
0 -> 2 assets backfilled, already-present assets skipped (no re-download),
second run uploads 0.
2026-06-23 23:06:43 +05:30
10 changed files with 1535 additions and 120 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.20.0",
"version": "3.20.4",
"engines": {
"bun": ">=1.2.9"
},
+245
View File
@@ -0,0 +1,245 @@
/**
* Unit tests for archiveGiteaRepo's return value and sanitizeRepoNameAlphaDashDot —
* regression coverage for #331's follow-up (repos falsely flagged as orphaned
* and archived, then unreachable by "Manual Sync" because the DB's
* mirroredLocation/name were never updated to the post-rename name).
*
* archiveGiteaRepo now reports the Gitea-side name it ended up with after a
* rename (mirror path) so callers (repository-cleanup-service.ts) can persist
* it, instead of leaving the DB pointing at a name that no longer exists.
*/
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
const mockHttpGet = mock(async (_url: string, _headers?: any) => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPatch = mock(async (_url: string, _body?: any, _headers?: any) => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPost = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpDelete = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const mockHttpPut = mock(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
class MockHttpError extends Error {
constructor(
message: string,
public status: number,
public statusText: string,
public response?: string
) {
super(message);
this.name = "HttpError";
}
}
mock.module("@/lib/http-client", () => ({
httpGet: mockHttpGet,
httpPatch: mockHttpPatch,
httpPost: mockHttpPost,
httpDelete: mockHttpDelete,
httpPut: mockHttpPut,
HttpError: MockHttpError,
}));
import { archiveGiteaRepo, sanitizeRepoNameAlphaDashDot } from "./gitea";
describe("sanitizeRepoNameAlphaDashDot", () => {
test("replaces disallowed characters with a dash", () => {
expect(sanitizeRepoNameAlphaDashDot("my repo!")).toBe("my-repo");
});
test("collapses consecutive disallowed characters into a single dash", () => {
expect(sanitizeRepoNameAlphaDashDot("a___b")).toBe("a-b");
});
test("trims leading and trailing separators/dots", () => {
expect(sanitizeRepoNameAlphaDashDot("--.foo.--")).toBe("foo");
});
test("leaves an already-valid AlphaDashDot name unchanged", () => {
expect(sanitizeRepoNameAlphaDashDot("valid-repo.name")).toBe("valid-repo.name");
});
});
describe("archiveGiteaRepo", () => {
const client = { url: "https://gitea.example.com", token: "test-token" };
let originalConsoleLog: typeof console.log;
let originalConsoleWarn: typeof console.warn;
let originalConsoleError: typeof console.error;
let originalConsoleDebug: typeof console.debug;
beforeEach(() => {
mockHttpGet.mockClear();
mockHttpPatch.mockClear();
mockHttpPost.mockClear();
mockHttpDelete.mockClear();
// Reset to benign defaults; individual tests override with mockImplementationOnce/mockImplementation.
mockHttpGet.mockImplementation(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementation(async () => ({
data: {},
status: 200,
statusText: "OK",
headers: new Headers(),
}));
originalConsoleLog = console.log;
originalConsoleWarn = console.warn;
originalConsoleError = console.error;
originalConsoleDebug = console.debug;
console.log = mock(() => {});
console.warn = mock(() => {});
console.error = mock(() => {});
console.debug = mock(() => {});
});
afterEach(() => {
console.log = originalConsoleLog;
console.warn = originalConsoleWarn;
console.error = originalConsoleError;
console.debug = originalConsoleDebug;
});
test("mirror repo rename returns the new archived name", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result).toEqual({ archivedName: "archived-my-repo" });
// Rename PATCH + mirror-interval-disable PATCH
expect(mockHttpPatch).toHaveBeenCalledTimes(2);
const renameCall = mockHttpPatch.mock.calls[0];
expect(String(renameCall[0])).toContain("/api/v1/repos/owner/my-repo");
expect(renameCall[1]).toMatchObject({ name: "archived-my-repo" });
});
test("already-archived mirror repo returns the existing name without re-renaming", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "archived-my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "archived-my-repo");
expect(result).toEqual({ archivedName: "archived-my-repo" });
expect(mockHttpPatch).not.toHaveBeenCalled();
});
test("non-mirror repo archives natively and returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "regular-repo", mirror: false, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementationOnce(async () => ({
data: { archived: true },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "regular-repo");
expect(result).toEqual({ archivedName: null });
expect(mockHttpPatch).toHaveBeenCalledTimes(1);
expect(mockHttpPatch.mock.calls[0][1]).toMatchObject({ archived: true });
});
test("rename PATCH failure (primary and timestamped fallback both fail) returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
mockHttpPatch.mockImplementation(async () => {
throw new MockHttpError("Unprocessable Entity", 422, "Unprocessable Entity");
});
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result).toEqual({ archivedName: null });
// Primary rename attempt + timestamped fallback attempt, no interval-disable call
expect(mockHttpPatch).toHaveBeenCalledTimes(2);
});
test("mirror repo rename recovers via timestamped fallback after a primary conflict", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: { name: "my-repo", mirror: true, description: "" },
status: 200,
statusText: "OK",
headers: new Headers(),
}));
let callCount = 0;
mockHttpPatch.mockImplementation(async (url: string, body?: any) => {
callCount++;
if (callCount === 1) {
// Primary rename attempt fails (e.g. AlphaDashDot conflict)
throw new MockHttpError("conflict", 422, "Unprocessable Entity");
}
// Fallback rename attempt and the interval-disable PATCH both succeed
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
const result = await archiveGiteaRepo(client, "owner", "my-repo");
expect(result.archivedName).toMatch(/^archived-\d{14}-my-repo$/);
expect(mockHttpPatch).toHaveBeenCalledTimes(3);
});
test("repository not found in Gitea returns archivedName: null", async () => {
mockHttpGet.mockImplementationOnce(async () => ({
data: null,
status: 200,
statusText: "OK",
headers: new Headers(),
}));
const result = await archiveGiteaRepo(client, "owner", "missing-repo");
expect(result).toEqual({ archivedName: null });
expect(mockHttpPatch).not.toHaveBeenCalled();
});
});
+380 -5
View File
@@ -19,15 +19,19 @@ const mockCreatePreSyncBundleBackup = mock(() =>
let mockShouldCreatePreSyncBackup = false;
let mockShouldBlockSyncOnBackupFailure = true;
// Mock the database module
// Mock the database module. Every db.update(...).set(payload) is captured in
// dbUpdateSetCalls so tests can assert on what got written (e.g. archived
// repos keeping status "archived" after a Manual Sync).
const dbUpdateSetCalls: any[] = [];
const mockDb = {
insert: mock((table: any) => ({
values: mock((data: any) => Promise.resolve({ insertedId: "mock-id" }))
})),
update: mock(() => ({
set: mock(() => ({
where: mock(() => Promise.resolve())
}))
set: mock((data: any) => {
dbUpdateSetCalls.push(data);
return { where: mock(() => Promise.resolve()) };
})
}))
};
@@ -173,10 +177,74 @@ const mockHttpGet = mock(async (url: string, headers?: any) => {
headers: new Headers(),
};
}
// Only reachable at the "archived-{name}" candidate — the base name
// ("starred/broken-repo") deliberately falls through to the generic 404
// below, simulating a repo that archiveGiteaRepo already renamed in Gitea.
// original_url matches the test repository's GitHub source, so the
// fallback candidate's source-identity guard accepts it.
if (url.includes("/api/v1/repos/starred/archived-broken-repo")) {
return {
data: {
id: 792,
name: "archived-broken-repo",
mirror: true,
owner: { login: "starred" },
mirror_interval: "0h",
original_url: "https://github.com/user/broken-repo.git",
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
// Collision scenario: this archived mirror belongs to a DIFFERENT GitHub
// source (otheruser/collide-repo) that happens to share the base name with
// the test repository (user/collide-repo). The base name
// ("starred/collide-repo") falls through to the generic 404 below, so the
// archived-{name} fallback candidate is the only match — and its
// original_url must cause the source-identity guard to reject it.
if (url.includes("/api/v1/repos/starred/archived-collide-repo")) {
return {
data: {
id: 793,
name: "archived-collide-repo",
mirror: true,
owner: { login: "starred" },
mirror_interval: "0h",
original_url: "https://github.com/otheruser/collide-repo.git",
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
// Simulates Forgejo silently following a 301 redirect for a renamed repo:
// a GET for the STALE (pre-rename) path returns 200 with the repo's
// CURRENT identity in the response body (name differs from what was
// requested), exactly as Bun's fetch behaves after following Forgejo's
// redirect for a repo renamed from "renamed-repo" to
// "archived-renamed-repo". See #331 follow-up / canonical-identity
// adoption in syncGiteaRepoEnhanced.
if (url.includes("/api/v1/repos/starred/renamed-repo")) {
return {
data: {
id: 891,
name: "archived-renamed-repo",
mirror: true,
owner: { login: "starred" },
private: false,
},
status: 200,
statusText: "OK",
headers: new Headers(),
};
}
if (url.includes("/api/v1/repos/")) {
throw new MockHttpError("Not Found", 404, "Not Found");
}
// Handle org GET requests based on test context
if (url.includes("/api/v1/orgs/starred")) {
orgCheckCount++;
@@ -239,10 +307,18 @@ const mockHttpDelete = mock(async (url: string, headers?: any) => {
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
// Observable so tests can assert that the mirror-interval PATCH is (not)
// issued — e.g. archived repos must never have Gitea's periodic pulling
// re-enabled by a Manual Sync.
const mockHttpPatch = mock(async (url: string, body?: any, headers?: any) => {
return { data: {}, status: 200, statusText: "OK", headers: new Headers() };
});
mock.module("@/lib/http-client", () => ({
httpGet: mockHttpGet,
httpPost: mockHttpPost,
httpDelete: mockHttpDelete,
httpPatch: mockHttpPatch,
HttpError: MockHttpError
}));
@@ -284,6 +360,8 @@ describe("Enhanced Gitea Operations", () => {
mockHttpGet.mockClear();
mockHttpPost.mockClear();
mockHttpDelete.mockClear();
mockHttpPatch.mockClear();
dbUpdateSetCalls.length = 0;
mockCreatePreSyncBundleBackup.mockClear();
mockCreatePreSyncBundleBackup.mockImplementation(() =>
Promise.resolve({ bundlePath: "/tmp/mock.bundle" })
@@ -612,6 +690,303 @@ describe("Enhanced Gitea Operations", () => {
expect(String(mirrorSyncCalls[0][0])).not.toContain("/api/v1/repos/ceph/test-repo/mirror-sync");
});
test("falls back to the archived-{name} candidate when repository.status is 'archived'", async () => {
// Regression for #331 follow-up: repos archived before mirroredLocation
// was backfilled on rename (or any lingering false-positive orphan hit)
// are unreachable by name/expected-owner alone once archiveGiteaRepo has
// renamed them in Gitea to `archived-{sanitized name}`. syncGiteaRepoEnhanced
// must still find them via "Manual Sync" without manual intervention.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoArchived1",
name: "broken-repo",
fullName: "user/broken-repo",
owner: "user",
cloneUrl: "https://github.com/user/broken-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
visibility: "public",
userId: "user123",
// No mirroredLocation recorded — this repo predates the DB backfill
// added alongside archiveGiteaRepo's new return value.
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-broken-repo/mirror-sync"
);
// The base (pre-archive) name must have been probed and rejected
// (404) before falling back to the archived-{name} candidate.
const repoInfoGets = mockHttpGet.mock.calls.filter((call) =>
String(call[0]).includes("/api/v1/repos/starred/")
);
expect(
repoInfoGets.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/broken-repo")
)
).toBe(true);
expect(
repoInfoGets.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/archived-broken-repo")
)
).toBe(true);
});
test("adopts canonical identity from response body when GET follows a stale-name redirect", async () => {
// Regression for #331 follow-up, verified end-to-end on Forgejo 15.0.3:
// when a repo has been renamed (e.g. by the orphan-archive flow, or
// manually by a user), Forgejo answers a GET for the OLD name with a
// 301 redirect to the new name. Bun's fetch follows it silently and
// returns 200 with the repo's CURRENT data in the body, while the
// code still has the stale name it requested. If syncGiteaRepoEnhanced
// kept using the requested (stale) name for the follow-up POST
// .../mirror-sync, that POST would hit the same 301, get its method
// downgraded to GET per the WHATWG redirect spec, and the POST-only
// endpoint would return 405. This must work for non-archived repos
// too — a user renaming a repo in Forgejo manually is the general case.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoRenamed1",
name: "renamed-repo",
fullName: "user/renamed-repo",
owner: "user",
cloneUrl: "https://github.com/user/renamed-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("mirrored"),
visibility: "public",
userId: "user123",
// Stale: recorded before the rename happened in Gitea/Forgejo.
mirroredLocation: "starred/renamed-repo",
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-renamed-repo/mirror-sync"
);
expect(String(mirrorSyncCalls[0][0])).not.toContain(
"/api/v1/repos/starred/renamed-repo/mirror-sync"
);
});
test("keeps archived repos archived and skips the mirror-interval PATCH on Manual Sync", async () => {
// Documented contract (AutomationSettings.tsx): "Archive renames mirror
// backups with an archived- prefix and disables automatic syncs—use
// Manual Sync when you want to refresh." A successful Manual Sync of an
// archived repo must therefore refresh once WITHOUT (a) flipping status
// to "synced" (which would re-enroll it into the scheduler's auto-sync
// pool), (b) clearing the archived errorMessage annotation, or
// (c) PATCHing the mirror interval (which would re-enable Forgejo's own
// periodic pulling that archiveGiteaRepo disabled).
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
// Would normally trigger the mirror-interval PATCH on every sync.
mirrorInterval: "8h",
},
};
const repository: Repository = {
id: "repoArchived2",
name: "broken-repo",
fullName: "user/broken-repo",
owner: "user",
url: "https://github.com/user/broken-repo",
cloneUrl: "https://github.com/user/broken-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
isArchived: true,
visibility: "public",
userId: "user123",
errorMessage: "Repository archived - no longer in GitHub",
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
// The mirror-sync itself must still happen (that's the point of
// Manual Sync on an archived repo).
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain(
"/api/v1/repos/starred/archived-broken-repo/mirror-sync"
);
// No mirror-interval PATCH despite config.giteaConfig.mirrorInterval.
expect(mockHttpPatch).not.toHaveBeenCalled();
// The success-path DB update (the one recording lastMirrored) keeps
// status "archived" and does not clear errorMessage.
const successUpdate = dbUpdateSetCalls.find((data) => "lastMirrored" in data);
expect(successUpdate).toBeDefined();
expect(successUpdate.status).toBe("archived");
expect("errorMessage" in successUpdate).toBe(false);
expect(successUpdate.mirroredLocation).toBe("starred/archived-broken-repo");
});
test("rejects an archived-{name} fallback candidate whose original_url points at a different source", async () => {
// Two sources sharing a base name: the user mirrors user/collide-repo,
// but starred/archived-collide-repo in Gitea is the archived mirror of
// otheruser/collide-repo. The guessed archived-{name} fallback must be
// rejected via its original_url instead of syncing (and rewriting the
// DB row of) the wrong repository. With every candidate exhausted, the
// sync fails with the not-found error.
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: false,
},
};
const repository: Repository = {
id: "repoCollide1",
name: "collide-repo",
fullName: "user/collide-repo",
owner: "user",
url: "https://github.com/user/collide-repo",
cloneUrl: "https://github.com/user/collide-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("archived"),
isArchived: true,
visibility: "public",
userId: "user123",
// No mirroredLocation — forces reliance on the guessed fallback.
createdAt: new Date(),
updatedAt: new Date(),
};
await expect(
syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
)
).rejects.toThrow("Repository collide-repo not found in Gitea. Tried locations:");
// The wrong repo must never receive a mirror-sync POST.
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(0);
// The fallback candidate WAS probed (and then rejected by the guard).
expect(
mockHttpGet.mock.calls.some((call) =>
String(call[0]).endsWith("/api/v1/repos/starred/archived-collide-repo")
)
).toBe(true);
});
test("blocks sync when pre-sync snapshot fails and blocking is enabled", async () => {
mockShouldCreatePreSyncBackup = true;
mockShouldBlockSyncOnBackupFailure = true;
+99 -6
View File
@@ -26,6 +26,7 @@ import {
strategyNeedsDetection,
} from "./repo-backup";
import { detectForcePush } from "./utils/force-push-detection";
import { sanitizeRepoNameAlphaDashDot } from "./gitea";
import {
parseRepositoryMetadataState,
serializeRepositoryMetadataState,
@@ -61,6 +62,27 @@ export interface GiteaRepoInfo {
interface SyncTargetCandidate {
owner: string;
repoName: string;
/**
* True for the guessed `archived-{name}` fallback candidate (see
* syncGiteaRepoEnhanced). Unlike the recorded mirroredLocation or the
* expected-owner candidate, this one is derived purely from the repo NAME,
* so it can collide with a different source repo that shares the same base
* name — it must pass an original_url source check before being accepted.
*/
isArchivedFallback?: boolean;
}
/**
* Normalize a git remote URL for source-identity comparison: lowercase,
* strip trailing slashes and a trailing `.git`.
*/
function normalizeSourceUrl(url: string): string {
return url
.trim()
.toLowerCase()
.replace(/\/+$/, "")
.replace(/\.git$/, "")
.replace(/\/+$/, "");
}
function parseMirroredLocation(location?: string | null): SyncTargetCandidate | null {
@@ -329,12 +351,32 @@ export async function syncGiteaRepoEnhanced({
// Resolve sync target in a backward-compatible order:
// 1) recorded mirroredLocation (actual historical mirror location)
// 2) owner derived from current strategy/config
// 3) (archived repos only) the `archived-{name}` rename that
// archiveGiteaRepo applies to mirror repos (see #331 follow-up).
// Repos archived before mirroredLocation was backfilled on rename, or
// any lingering false-positive orphan hit, would otherwise be
// unreachable from "Manual Sync" — the UI's documented way to refresh
// an archived mirror — because the recorded/expected name no longer
// exists and Gitea returns HTTP 405 for it.
const dependencies = deps ?? (await import("./gitea"));
const expectedOwner = await dependencies.getGiteaRepoOwnerAsync({ config, repository });
const recordedTarget = parseMirroredLocation(repository.mirroredLocation);
// Archived state can live in either field: the cleanup service sets both,
// but a failed retry can clobber `status` while `isArchived` survives.
const isArchivedRepo =
repository.status === "archived" || !!repository.isArchived;
const candidateTargets = dedupeSyncTargets([
...(recordedTarget ? [recordedTarget] : []),
{ owner: expectedOwner, repoName: repository.name },
...(isArchivedRepo
? [
{
owner: expectedOwner,
repoName: `archived-${sanitizeRepoNameAlphaDashDot(repository.name)}`,
isArchivedFallback: true,
},
]
: []),
]);
let repoOwner = expectedOwner;
@@ -360,8 +402,45 @@ export async function syncGiteaRepoEnhanced({
continue;
}
repoOwner = target.owner;
repoName = target.repoName;
// The archived-{name} fallback candidate is guessed from the repo NAME
// alone, so it can hit a DIFFERENT source's archived mirror when two
// sources share a base name (e.g. foo/tools and bar/tools both mirrored;
// foo's archived as `archived-tools`). Before accepting it, verify the
// candidate's original_url (the authoritative migration source — see
// GiteaRepoInfo) points at THIS repository's GitHub source. An empty/
// unset original_url is accepted as before (some migrations leave it
// unset). This guard deliberately does NOT apply to the recorded
// mirroredLocation or expected-name candidates.
if (
target.isArchivedFallback &&
typeof candidateInfo.original_url === "string" &&
candidateInfo.original_url.trim() !== ""
) {
const candidateSource = normalizeSourceUrl(candidateInfo.original_url);
const ownSources = [repository.cloneUrl, repository.url]
.filter((u): u is string => typeof u === "string" && u.trim() !== "")
.map(normalizeSourceUrl);
if (!ownSources.includes(candidateSource)) {
console.warn(
`[Sync] Skipping archived-name candidate ${target.owner}/${target.repoName} for ${repository.name}: its original_url (${candidateInfo.original_url}) points at a different source`
);
continue;
}
}
// Adopt the canonical identity from the response, not the requested target.
// Gitea/Forgejo answer GETs for a renamed repo's old name with a 301 that
// fetch follows silently, so `target` may be a stale pre-rename name; a
// follow-up POST (mirror-sync) to the stale path gets its method downgraded
// by the redirect and fails with 405. The response body always carries the
// repo's current name/owner (#331 follow-up, verified on Forgejo 15).
const canonicalOwner =
typeof candidateInfo.owner === "string"
? candidateInfo.owner
: candidateInfo.owner?.login;
repoOwner = canonicalOwner || target.owner;
repoName = candidateInfo.name || target.repoName;
repoInfo = candidateInfo;
break;
}
@@ -621,7 +700,13 @@ export async function syncGiteaRepoEnhanced({
// NOTE: Gitea/Forgejo's PATCH /repos/{owner}/{repo} API does not support
// updating mirror credentials (mirror_username/mirror_password). Repos that
// were originally migrated without credentials must be deleted and re-mirrored.
if (config.giteaConfig?.mirrorInterval) {
//
// Skipped for archived repos: archiveGiteaRepo deliberately disabled
// Gitea's own periodic pulling, and the documented contract
// (AutomationSettings.tsx: "Archive ... disables automatic syncs—use
// Manual Sync when you want to refresh") says a Manual Sync refreshes
// once without re-enabling any automatic syncing.
if (config.giteaConfig?.mirrorInterval && !isArchivedRepo) {
try {
console.log(`[Sync] Updating mirror interval for ${repoOwner}/${repoName} to ${config.giteaConfig.mirrorInterval}`);
const updateUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}`;
@@ -844,14 +929,22 @@ export async function syncGiteaRepoEnhanced({
metadataState.lastSyncedAt = new Date().toISOString();
}
// Mark repo as "synced" in DB
// Mark repo as "synced" in DB — unless it's archived. The documented
// contract (AutomationSettings.tsx: "Archive ... disables automatic
// syncs—use Manual Sync when you want to refresh") means a Manual Sync
// of an archived repo must NOT re-enroll it into the scheduler's
// auto-sync pool (which selects mirrored/synced/failed/pending) nor
// clear the archived errorMessage annotation shown in the UI; it still
// records lastMirrored and the (possibly corrected) mirroredLocation.
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("synced"),
status: isArchivedRepo
? repoStatusEnum.parse("archived")
: repoStatusEnum.parse("synced"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
...(isArchivedRepo ? {} : { errorMessage: null }),
mirroredLocation: `${repoOwner}/${repoName}`,
metadata: metadataUpdated
? serializeRepositoryMetadataState(metadataState)
+52
View File
@@ -0,0 +1,52 @@
/**
* Regression test for the #334 sibling bug — labels silently dropped on issue update.
*
* Gitea/Forgejo's `EditIssueOption` has no `labels` field (only `CreateIssueOption`
* does), so a `labels` key in a `PATCH .../issues/{index}` body is silently ignored.
* The old code put `labels` in the update PATCH, so label changes never propagated
* onto already-mirrored issues. The fix builds the edit body WITHOUT labels
* (`buildGiteaIssueEditPayload`) and reconciles labels separately through the
* sub-resource `PUT .../issues/{index}/labels` (`buildGiteaIssueLabelsPayload`).
*
* Verified live against Gitea 1.24.7: PATCH with `labels` leaves the issue's labels
* unchanged; PUT to the labels sub-resource replaces them. Confirmed end-to-end that
* a drifted (label-less) mirrored issue reconciles back to its GitHub label set.
*/
import { describe, test, expect } from "bun:test";
import { buildGiteaIssueEditPayload, buildGiteaIssueLabelsPayload } from "@/lib/gitea";
describe("buildGiteaIssueEditPayload (#334 sibling)", () => {
test("edit body never carries `labels` (Gitea's EditIssueOption ignores it)", () => {
const payload = buildGiteaIssueEditPayload({
title: "[GH-ISSUE #1] Fix the thing",
body: "desc",
closed: false,
});
expect(payload).not.toHaveProperty("labels");
expect(payload).toEqual({
title: "[GH-ISSUE #1] Fix the thing",
body: "desc",
state: "open",
});
});
test("maps the closed flag to Gitea's `state`", () => {
expect(buildGiteaIssueEditPayload({ title: "t", body: "b", closed: true }).state).toBe("closed");
expect(buildGiteaIssueEditPayload({ title: "t", body: "b", closed: false }).state).toBe("open");
});
});
describe("buildGiteaIssueLabelsPayload (#334 sibling)", () => {
test("replaces the full label set with the resolved Gitea label ids", () => {
expect(buildGiteaIssueLabelsPayload([7, 9])).toEqual({ labels: [7, 9] });
});
test("sends an empty set so upstream label removals propagate", () => {
expect(buildGiteaIssueLabelsPayload([])).toEqual({ labels: [] });
});
test("treats a missing id list as an empty set (defensive)", () => {
expect(buildGiteaIssueLabelsPayload(undefined as any)).toEqual({ labels: [] });
});
});
+51
View File
@@ -0,0 +1,51 @@
/**
* Regression test for #334 — "Release titles not being mirrored properly".
*
* Root cause: the release create/update payloads sent the release title under the
* JSON key `title`, but Gitea/Forgejo's release API expects `name` (the API Go
* struct is `Title string \`json:"name"\``). `title` is silently dropped, so every
* mirrored release landed with a blank name.
*
* `buildGiteaReleasePayload` is the single source of truth for both the create
* (POST) and update (PATCH) bodies. Verified live against Gitea 1.24.7: a payload
* with `title` yields `name: ""`; a payload with `name` sets the title correctly.
*/
import { describe, test, expect } from "bun:test";
import { buildGiteaReleasePayload } from "@/lib/gitea";
describe("buildGiteaReleasePayload (#334)", () => {
test("carries the release title under `name`, never `title`", () => {
const payload = buildGiteaReleasePayload(
{ tag_name: "v0.19.0", name: "v0.19.0", draft: false, prerelease: false },
"## Features\n- something"
);
expect(payload.name).toBe("v0.19.0");
expect(payload).not.toHaveProperty("title");
expect(payload).toEqual({
tag_name: "v0.19.0",
name: "v0.19.0",
body: "## Features\n- something",
draft: false,
prerelease: false,
});
});
test("falls back to tag_name when the GitHub release name is empty or null", () => {
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3", name: null }, "x").name).toBe("v1.2.3");
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3", name: "" }, "x").name).toBe("v1.2.3");
expect(buildGiteaReleasePayload({ tag_name: "v1.2.3" }, "x").name).toBe("v1.2.3");
});
test("passes draft/prerelease/body through unchanged", () => {
const payload = buildGiteaReleasePayload(
{ tag_name: "v2.0.0", name: "Two", draft: true, prerelease: true },
"notes body"
);
expect(payload.body).toBe("notes body");
expect(payload.draft).toBe(true);
expect(payload.prerelease).toBe(true);
expect(payload.tag_name).toBe("v2.0.0");
});
});
+83 -1
View File
@@ -27,7 +27,10 @@
*/
import { describe, expect, it } from "bun:test";
import { classifyReleasesForReconciliation } from "@/lib/gitea";
import {
classifyReleasesForReconciliation,
classifyAssetsForReconciliation,
} from "@/lib/gitea";
describe("classifyReleasesForReconciliation", () => {
describe("normal repo — published_at order matches tag-commit order", () => {
@@ -148,3 +151,82 @@ describe("classifyReleasesForReconciliation", () => {
});
});
});
/**
* Asset reconciliation — regression for #331.
*
* Root cause: assets were uploaded only on the create path. When a Gitea release
* already existed (every re-sync, or after an interrupted first upload), the update
* path PATCHed the body and `continue`d without ever touching assets — so a release
* that existed without its full asset set stayed permanently asset-less and re-syncing
* could never heal it. Reproduced on a real Forgejo pull-mirror: GitHub release with
* two 35-40MB binaries → Gitea release with 0 assets → re-sync logged "Updating
* existing release" and left it at 0.
*
* Fix: reconcile assets idempotently on both paths via classifyAssetsForReconciliation.
*/
describe("classifyAssetsForReconciliation", () => {
it("uploads all assets when the Gitea release has none (the #331 broken state)", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea: Array<{ id: number; name: string; size: number }> = [];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([
{ name: "base.zip", replaceAssetId: null },
{ name: "extras.zip", replaceAssetId: null },
]);
});
it("backfills only the missing asset when one already exists", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [{ id: 9, name: "base.zip", size: 40_264_954 }];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual(["base.zip"]);
expect(toUpload).toEqual([{ name: "extras.zip", replaceAssetId: null }]);
});
it("is idempotent — skips everything when all assets already match by name+size", () => {
const github = [
{ name: "base.zip", size: 40_264_954 },
{ name: "extras.zip", size: 37_098_528 },
];
const gitea = [
{ id: 9, name: "base.zip", size: 40_264_954 },
{ id: 10, name: "extras.zip", size: 37_098_528 },
];
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual(["base.zip", "extras.zip"]);
});
it("replaces an asset whose size changed upstream (re-upload over the stale copy)", () => {
const github = [{ name: "firmware.bin", size: 2048 }];
const gitea = [{ id: 42, name: "firmware.bin", size: 1024 }]; // truncated/stale
const { toUpload, toSkip } = classifyAssetsForReconciliation(github, gitea);
expect(toSkip).toEqual([]);
expect(toUpload).toEqual([{ name: "firmware.bin", replaceAssetId: 42 }]);
});
it("handles a release with no GitHub assets", () => {
const { toUpload, toSkip } = classifyAssetsForReconciliation(
[],
[{ id: 1, name: "leftover.zip", size: 10 }]
);
expect(toUpload).toEqual([]);
expect(toSkip).toEqual([]);
});
});
+429 -98
View File
@@ -2186,6 +2186,98 @@ export const syncGiteaRepo = async ({
}
};
/**
* Build the JSON body for creating/updating a Gitea release.
*
* Gitea/Forgejo expose the release title through the JSON field `name`, not
* `title` (the API Go struct is `Title string \`json:"name"\``); sending `title`
* is silently ignored and leaves the release name blank (#334). Create and update
* send the same fields; `target` is intentionally omitted (see #331/#333) so Gitea
* attaches the release to the already-synced tag instead of 404-ing on the target.
*/
export function buildGiteaReleasePayload(
release: { tag_name: string; name?: string | null; draft?: boolean; prerelease?: boolean },
releaseNote: string
): { tag_name: string; name: string; body: string; draft?: boolean; prerelease?: boolean } {
return {
tag_name: release.tag_name,
name: release.name || release.tag_name,
body: releaseNote,
draft: release.draft,
prerelease: release.prerelease,
};
}
/**
* Build the JSON body for a Gitea issue / PR-as-issue edit (PATCH .../issues/{index}).
*
* Deliberately excludes `labels`: Gitea's `EditIssueOption` has no `labels` field
* (only `CreateIssueOption` does), so any `labels` key here is silently dropped —
* the same class of bug as the release `title` mix-up (#334 sibling). Labels are
* applied separately via the labels sub-resource (see buildGiteaIssueLabelsPayload).
*/
export function buildGiteaIssueEditPayload(opts: {
title: string;
body: string;
closed: boolean;
}): { title: string; body: string; state: "open" | "closed" } {
return { title: opts.title, body: opts.body, state: opts.closed ? "closed" : "open" };
}
/**
* Build the JSON body for the Gitea issue labels sub-resource
* (PUT .../issues/{index}/labels), which replaces the full label set idempotently.
*/
export function buildGiteaIssueLabelsPayload(labelIds: number[]): { labels: number[] } {
return { labels: labelIds ?? [] };
}
/**
* Replace the label set on an existing Gitea issue (or PR-as-issue) via the
* dedicated labels sub-resource.
*
* Gitea/Forgejo's `EditIssueOption` has no `labels` field (only
* `CreateIssueOption` does), so a `labels` key in a `PATCH .../issues/{index}`
* body is silently dropped by the JSON decoder — the same class of bug as the
* release `title` vs `name` mix-up (#334). Label changes on an already-mirrored
* issue therefore have to go through `PUT .../issues/{index}/labels`, which
* replaces the whole set idempotently: it both applies newly added labels and
* removes ones deleted upstream.
*
* Best-effort: labels are secondary metadata, so a transient failure here is
* logged and left to self-heal on the next sync rather than failing (and
* retrying) the entire issue + comment mirror.
*/
async function reconcileGiteaIssueLabels({
config,
decryptedConfig,
giteaOwner,
repoName,
issueNumber,
labelIds,
}: {
config: Partial<Config>;
decryptedConfig: Config;
giteaOwner: string;
repoName: string;
issueNumber: number;
labelIds: number[];
}): Promise<void> {
try {
await httpPut(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${issueNumber}/labels`,
buildGiteaIssueLabelsPayload(labelIds),
{ Authorization: `token ${decryptedConfig.giteaConfig!.token}` }
);
} catch (error) {
console.warn(
`[Labels] Failed to reconcile labels on issue #${issueNumber}: ${
error instanceof Error ? error.message : String(error)
} (will retry on next sync)`
);
}
}
export const mirrorGitRepoIssuesToGitea = async ({
config,
octokit,
@@ -2437,12 +2529,11 @@ export const mirrorGitRepoIssuesToGitea = async ({
targetIssueNumber = existingIssue.number;
await httpPatch(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}`,
{
buildGiteaIssueEditPayload({
title: issuePayload.title,
body: issuePayload.body,
state: issue.state === "closed" ? "closed" : "open",
labels: issuePayload.labels,
},
closed: issue.state === "closed",
}),
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
}
@@ -2487,12 +2578,11 @@ export const mirrorGitRepoIssuesToGitea = async ({
);
await httpPatch(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}`,
{
buildGiteaIssueEditPayload({
title: issuePayload.title,
body: issuePayload.body,
state: issue.state === "closed" ? "closed" : "open",
labels: issuePayload.labels,
},
closed: issue.state === "closed",
}),
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
}
@@ -2531,6 +2621,21 @@ export const mirrorGitRepoIssuesToGitea = async ({
}
}
// Gitea's EditIssueOption ignores `labels`, so the PATCH above can't change
// them on an already-mirrored issue — reconcile via the labels sub-resource.
// Only needed on the update paths; a freshly POSTed issue already got its
// labels from CreateIssueOption. (#334 sibling)
if (existingIssue) {
await reconcileGiteaIssueLabels({
config,
decryptedConfig,
giteaOwner,
repoName,
issueNumber: targetIssueNumber,
labelIds: giteaLabelIds,
});
}
// Clone comments
const comments = await octokit.paginate(
octokit.rest.issues.listComments,
@@ -2691,6 +2796,168 @@ export function classifyReleasesForReconciliation(
return { toCreate, toSkip };
}
/**
* Decide which of a GitHub release's assets need (re)uploading to Gitea.
*
* Compared by name:
* - present in Gitea with a matching size -> skip (already mirrored)
* - present with a different size -> upload, replacing the stale copy
* - absent -> upload (fresh)
*
* Pure function so the create/update reconciliation decision is unit-testable
* without hitting the network (regression guard for #331).
*/
export function classifyAssetsForReconciliation(
githubAssets: Array<{ name: string; size: number }>,
giteaAssets: Array<{ id: number; name: string; size: number }>
): {
toUpload: Array<{ name: string; replaceAssetId: number | null }>;
toSkip: string[];
} {
const existingByName = new Map(giteaAssets.map((a) => [a.name, a]));
const toUpload: Array<{ name: string; replaceAssetId: number | null }> = [];
const toSkip: string[] = [];
for (const asset of githubAssets) {
const existing = existingByName.get(asset.name);
if (existing && existing.size === asset.size) {
toSkip.push(asset.name);
} else {
toUpload.push({ name: asset.name, replaceAssetId: existing ? existing.id : null });
}
}
return { toUpload, toSkip };
}
/**
* Idempotently mirror a GitHub release's assets onto the matching Gitea release.
*
* Runs on BOTH the create and update paths so assets are reconciled on every sync,
* not only on the single sync where the Gitea release is first created. Previously
* assets were uploaded inline in the create path only; the update path PATCHed the
* body and `continue`d without ever looking at assets. Any release whose assets
* failed or were interrupted on first creation therefore stayed permanently
* asset-less, and re-syncing could never heal it (#331).
*
* Strategy: compare by asset name. Skip assets already present with a matching size;
* (re)upload anything missing, and replace an existing asset whose size differs
* (truncated/changed upstream). Returns per-release counts so the caller can report.
*/
async function reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId,
githubAssets,
tagName,
}: {
config: Partial<Config>;
decryptedConfig: Config;
repoOwner: string;
repoName: string;
giteaReleaseId: number;
githubAssets: Array<{ name: string; size: number; browser_download_url: string }>;
tagName: string;
}): Promise<{ uploaded: number; failed: number; skipped: number }> {
let uploaded = 0;
let failed = 0;
let skipped = 0;
if (!githubAssets || githubAssets.length === 0) {
return { uploaded, failed, skipped };
}
const giteaBaseUrl = config.giteaConfig!.url;
const giteaAuth = { Authorization: `token ${decryptedConfig.giteaConfig!.token}` };
// Fetch existing attachments so we only transfer what's missing or changed.
const existingAssets: Array<{ id: number; name: string; size: number }> = await httpGet(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets`,
giteaAuth
)
.then((r) => (Array.isArray(r?.data) ? r.data : []))
.catch(() => []);
const { toUpload, toSkip } = classifyAssetsForReconciliation(
githubAssets,
existingAssets
);
skipped = toSkip.length;
const githubByName = new Map(githubAssets.map((a) => [a.name, a]));
for (const { name, replaceAssetId } of toUpload) {
const asset = githubByName.get(name)!;
try {
// Download from GitHub. fetch strips the Authorization header on the
// cross-host redirect to GitHub's object storage, so this works for both
// public and private release assets.
console.log(
`[Releases] Downloading asset: ${asset.name} (${asset.size} bytes) for ${tagName}`
);
const assetResponse = await fetch(asset.browser_download_url, {
headers: {
Accept: "application/octet-stream",
Authorization: `token ${decryptedConfig.githubConfig!.token}`,
},
});
if (!assetResponse.ok) {
console.error(
`[Releases] Failed to download asset ${asset.name}: ${assetResponse.status} ${assetResponse.statusText}`
);
failed++;
continue;
}
const assetData = await assetResponse.arrayBuffer();
// Gitea rejects a duplicate attachment name, so drop a stale/mismatched
// copy before re-uploading.
if (replaceAssetId !== null) {
await httpDelete(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets/${replaceAssetId}`,
giteaAuth
).catch(() => null);
}
const formData = new FormData();
formData.append("attachment", new Blob([assetData]), asset.name);
const uploadResponse = await fetch(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets?name=${encodeURIComponent(asset.name)}`,
{
method: "POST",
headers: { Authorization: `token ${decryptedConfig.giteaConfig!.token}` },
body: formData,
}
);
if (uploadResponse.ok) {
console.log(`[Releases] Successfully uploaded asset: ${asset.name}`);
uploaded++;
} else {
const errorText = await uploadResponse.text();
console.error(
`[Releases] Failed to upload asset ${asset.name}: ${uploadResponse.status} ${errorText}`
);
failed++;
}
} catch (assetError) {
console.error(
`[Releases] Error processing asset ${asset.name}: ${
assetError instanceof Error ? assetError.message : String(assetError)
}`
);
failed++;
}
}
return { uploaded, failed, skipped };
}
export async function mirrorGitHubReleasesToGitea({
octokit,
repository,
@@ -2719,6 +2986,14 @@ export async function mirrorGitHubReleasesToGitea({
const repoOwner = giteaOwner || (await getGiteaRepoOwnerAsync({ config, repository }));
const repoName = giteaRepoName || repository.name;
// Derive GITHUB coordinates from fullName, matching the issues/PRs/labels/
// milestones mirror functions (`const [owner, repo] = repository.fullName.split("/")`).
// repository.name/owner can drift from the GitHub source (e.g. Gitea-side
// renames), so fullName is authoritative; fall back only if it's malformed.
const [fullNameOwner, fullNameRepo] = (repository.fullName || "").split("/");
const githubOwner = fullNameOwner && fullNameRepo ? fullNameOwner : repository.owner;
const githubRepo = fullNameOwner && fullNameRepo ? fullNameRepo : repository.name;
// Verify the repository exists in Gitea before attempting to mirror releases
console.log(`[Releases] Verifying repository ${repoName} exists at ${repoOwner}`);
const repoExists = await isRepoPresentInGitea({
@@ -2744,8 +3019,8 @@ export async function mirrorGitHubReleasesToGitea({
while (releases.length < releaseLimit) {
const response = await octokit.rest.repos.listReleases({
owner: repository.owner,
repo: repository.name,
owner: githubOwner,
repo: githubRepo,
per_page: perPage,
page,
});
@@ -2776,6 +3051,9 @@ export async function mirrorGitHubReleasesToGitea({
let mirroredCount = 0;
let skippedCount = 0;
let skippedMissingTagCount = 0;
let totalAssetsUploaded = 0;
let totalAssetsFailed = 0;
// Process releases in their GitHub API order (newest first by default)
const releasesToProcess = limitedReleases.slice();
@@ -2830,14 +3108,7 @@ export async function mirrorGitHubReleasesToGitea({
await httpPatch(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
{
tag_name: release.tag_name,
target: release.target_commitish,
title: release.name || release.tag_name,
body: releaseNote,
draft: release.draft,
prerelease: release.prerelease,
},
buildGiteaReleasePayload(release, releaseNote),
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -2853,6 +3124,49 @@ export async function mirrorGitHubReleasesToGitea({
console.log(`[Releases] Release ${release.tag_name} already up-to-date, skipping`);
skippedCount++;
}
// Reconcile assets on every sync — backfill any that are missing or changed.
// The update path used to `continue` here without touching assets, so a
// release that existed without its full asset set stayed broken forever (#331).
const assetResult = await reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId: existingRelease.id,
githubAssets: release.assets || [],
tagName: release.tag_name,
});
if (assetResult.uploaded > 0) {
console.log(
`[Releases] Backfilled ${assetResult.uploaded} missing/changed asset(s) for existing release ${release.tag_name}`
);
}
totalAssetsUploaded += assetResult.uploaded;
totalAssetsFailed += assetResult.failed;
continue;
}
// The git tag must already exist in Gitea before we create a release for it.
// For a mirror, tags are synced from upstream by Gitea's own git mirror, which
// can lag behind this metadata sync (e.g. a large/slow initial clone). If the
// tag isn't present yet, skip and let a later sync pick it up — do NOT ask Gitea
// to create the release against a `target` branch:
// - if the target can't be resolved Gitea returns 404 "The target couldn't be
// found" and the release is lost (#331),
// - if it can, Gitea would create a brand-new tag at the wrong commit.
const tagExists = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/tags/${encodeURIComponent(release.tag_name)}`,
{ Authorization: `token ${decryptedConfig.giteaConfig.token}` }
)
.then(() => true)
.catch(() => false);
if (!tagExists) {
console.warn(
`[Releases] Tag ${release.tag_name} is not present in Gitea yet — skipping release for now (the git mirror may still be syncing; it will be retried on the next sync)`
);
skippedMissingTagCount++;
continue;
}
@@ -2862,71 +3176,31 @@ export async function mirrorGitHubReleasesToGitea({
} else {
console.log(`[Releases] Creating release ${release.tag_name} with GitHub date header (no changelog)`);
}
const createReleaseResponse = await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases`,
{
tag_name: release.tag_name,
target: release.target_commitish,
title: release.name || release.tag_name,
body: releaseNote,
draft: release.draft,
prerelease: release.prerelease,
},
buildGiteaReleasePayload(release, releaseNote),
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
);
// Mirror release assets if they exist
// Mirror release assets if they exist (idempotent — see reconcileReleaseAssets)
if (release.assets && release.assets.length > 0) {
console.log(`[Releases] Mirroring ${release.assets.length} assets for release ${release.tag_name}`);
for (const asset of release.assets) {
try {
// Download the asset from GitHub
console.log(`[Releases] Downloading asset: ${asset.name} (${asset.size} bytes)`);
const assetResponse = await fetch(asset.browser_download_url, {
headers: {
'Accept': 'application/octet-stream',
'Authorization': `token ${decryptedConfig.githubConfig.token}`,
},
});
if (!assetResponse.ok) {
console.error(`[Releases] Failed to download asset ${asset.name}: ${assetResponse.statusText}`);
continue;
}
const assetData = await assetResponse.arrayBuffer();
// Upload the asset to Gitea release
const formData = new FormData();
formData.append('attachment', new Blob([assetData]), asset.name);
const uploadResponse = await fetch(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${createReleaseResponse.data.id}/assets?name=${encodeURIComponent(asset.name)}`,
{
method: 'POST',
headers: {
'Authorization': `token ${decryptedConfig.giteaConfig.token}`,
},
body: formData,
}
);
if (uploadResponse.ok) {
console.log(`[Releases] Successfully uploaded asset: ${asset.name}`);
} else {
const errorText = await uploadResponse.text();
console.error(`[Releases] Failed to upload asset ${asset.name}: ${errorText}`);
}
} catch (assetError) {
console.error(`[Releases] Error processing asset ${asset.name}: ${assetError instanceof Error ? assetError.message : String(assetError)}`);
}
}
const assetResult = await reconcileReleaseAssets({
config,
decryptedConfig,
repoOwner,
repoName,
giteaReleaseId: createReleaseResponse.data.id,
githubAssets: release.assets,
tagName: release.tag_name,
});
totalAssetsUploaded += assetResult.uploaded;
totalAssetsFailed += assetResult.failed;
}
mirroredCount++;
const noteInfo = originalReleaseNote ? ` with ${originalReleaseNote.length} character changelog` : " without changelog";
console.log(`[Releases] Successfully mirrored release: ${release.tag_name}${noteInfo}`);
@@ -2935,7 +3209,21 @@ export async function mirrorGitHubReleasesToGitea({
}
}
console.log(`✅ Mirrored/Updated ${mirroredCount} releases to Gitea (${skippedCount} already up-to-date)`);
console.log(
`✅ Mirrored/Updated ${mirroredCount} releases to Gitea (${skippedCount} already up-to-date, ${skippedMissingTagCount} skipped: tag not synced yet); assets uploaded: ${totalAssetsUploaded}, failed: ${totalAssetsFailed}`
);
if (skippedMissingTagCount > 0) {
console.warn(
`[Releases] ${skippedMissingTagCount} release(s) skipped because their git tag is not in Gitea yet for ${repository.fullName} — these will be created automatically once the git mirror finishes syncing the tags`
);
}
if (totalAssetsFailed > 0) {
console.error(
`[Releases] ⚠️ ${totalAssetsFailed} release asset(s) failed to mirror for ${repository.fullName} — they will be retried on the next sync`
);
}
// Enforce release retention limit by removing the oldest excess releases from Gitea
try {
@@ -3279,12 +3567,11 @@ export async function mirrorGitRepoPullRequestsToGitea({
if (existingPrIssue) {
await httpPatch(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${existingPrIssue.number}`,
{
buildGiteaIssueEditPayload({
title: issueData.title,
body: issueData.body,
state: issueData.closed ? "closed" : "open",
labels: issueData.labels,
},
closed: issueData.closed,
}),
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
}
@@ -3323,6 +3610,20 @@ export async function mirrorGitRepoPullRequestsToGitea({
}
}
// Gitea drops `labels` on issue edit, so the "pull-request" marker label
// can't be set via the PATCH above — reconcile it on the update path.
// (#334 sibling)
if (existingPrIssue) {
await reconcileGiteaIssueLabels({
config,
decryptedConfig,
giteaOwner,
repoName,
issueNumber: existingPrIssue.number,
labelIds: issueData.labels,
});
}
successCount++;
console.log(`[Pull Requests] ✅ Successfully created issue for PR #${pr.number}`);
} catch (apiError) {
@@ -3367,12 +3668,11 @@ export async function mirrorGitRepoPullRequestsToGitea({
if (existingPrIssue) {
await httpPatch(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${existingPrIssue.number}`,
{
buildGiteaIssueEditPayload({
title: basicIssueData.title,
body: basicIssueData.body,
state: basicIssueData.closed ? "closed" : "open",
labels: basicIssueData.labels,
},
closed: basicIssueData.closed,
}),
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
}
@@ -3410,6 +3710,19 @@ export async function mirrorGitRepoPullRequestsToGitea({
}
}
// Same as the enriched path — reconcile the marker label via the labels
// sub-resource since PATCH ignores it. (#334 sibling)
if (existingPrIssue) {
await reconcileGiteaIssueLabels({
config,
decryptedConfig,
giteaOwner,
repoName,
issueNumber: existingPrIssue.number,
labelIds: basicIssueData.labels,
});
}
successCount++;
console.log(`[Pull Requests] ✅ Created basic issue for PR #${pr.number}`);
} catch (error) {
@@ -3739,29 +4052,43 @@ export async function deleteGiteaRepo(
}
}
/**
* Sanitize a repository name to satisfy Gitea's AlphaDashDot rule for repo
* names (letters, digits, `.`, and `-`; no leading/trailing separators).
*
* Extracted from archiveGiteaRepo (module-level, exported) so gitea-enhanced.ts
* can build the identical `archived-{name}` candidate when probing for a repo
* that was renamed by this exact archive flow — either just now, or by an
* older version of this code before mirroredLocation was backfilled on rename
* (see #331 follow-up).
*/
export function sanitizeRepoNameAlphaDashDot(name: string): string {
// Replace anything that's not [A-Za-z0-9.-] with '-'
const base = name.replace(/[^A-Za-z0-9.-]+/g, "-").replace(/-+/g, "-");
// Trim leading/trailing separators and dots for safety
return base.replace(/^[.-]+/, "").replace(/[.-]+$/, "");
}
/**
* Archive a repository in Gitea
*
*
* IMPORTANT: This function NEVER deletes data. It only marks repositories as archived.
* - For regular repos: Uses Gitea's archive feature (makes read-only)
* - For mirror repos: Renames with [ARCHIVED] prefix (Gitea doesn't allow archiving mirrors)
*
*
* This ensures backups are preserved even when the GitHub source disappears.
*
* Returns the Gitea-side name the repository ended up with after a rename
* (mirror path), or `null` when no rename occurred (regular-repo archive
* path, or any failure). Callers that persist `mirroredLocation` should only
* update it when `archivedName` is non-null.
*/
export async function archiveGiteaRepo(
client: { url: string; token: string },
owner: string,
repo: string
): Promise<void> {
): Promise<{ archivedName: string | null }> {
try {
// Helper: sanitize to Gitea's AlphaDashDot rule
const sanitizeRepoNameAlphaDashDot = (name: string): string => {
// Replace anything that's not [A-Za-z0-9.-] with '-'
const base = name.replace(/[^A-Za-z0-9.-]+/g, "-").replace(/-+/g, "-");
// Trim leading/trailing separators and dots for safety
return base.replace(/^[.-]+/, "").replace(/[.-]+$/, "");
};
// First, check if this is a mirror repository
const repoResponse = await httpGet(
`${client.url}/api/v1/repos/${owner}/${repo}`,
@@ -3772,7 +4099,7 @@ export async function archiveGiteaRepo(
if (!repoResponse.data) {
console.warn(`[Archive] Repository ${owner}/${repo} not found in Gitea. Skipping.`);
return;
return { archivedName: null };
}
if (repoResponse.data?.mirror) {
@@ -3794,7 +4121,7 @@ export async function archiveGiteaRepo(
normalizedName.startsWith('archived-')
) {
console.log(`[Archive] Repository ${owner}/${repo} already marked as archived. Skipping.`);
return;
return { archivedName: currentName };
}
// Use a safe prefix and sanitize the name to satisfy AlphaDashDot rule
@@ -3839,7 +4166,7 @@ export async function archiveGiteaRepo(
// If this also fails, log but don't throw - data remains preserved
console.error(`[Archive] Failed to rename mirror repository ${owner}/${repo}:`, e2);
console.log(`[Archive] Repository ${owner}/${repo} remains accessible but not marked as archived`);
return;
return { archivedName: null };
}
}
@@ -3863,6 +4190,8 @@ export async function archiveGiteaRepo(
// Non-critical - repo is still preserved even if we can't change interval
console.debug(`[Archive] Could not disable mirror interval (non-critical):`, intervalError);
}
return { archivedName };
} else {
// For non-mirror repositories, use Gitea's native archive feature
// This makes the repository read-only but preserves all data
@@ -3883,10 +4212,11 @@ export async function archiveGiteaRepo(
// If archive fails, log but data is still preserved in Gitea
console.error(`[Archive] Failed to archive repository ${owner}/${repo}: ${response.status}`);
console.log(`[Archive] Repository ${owner}/${repo} remains accessible but not marked as archived`);
return;
return { archivedName: null };
}
console.log(`[Archive] Successfully archived repository ${owner}/${repo} (now read-only)`);
return { archivedName: null };
}
} catch (error) {
// Even on error, the repository data is preserved in Gitea
@@ -3894,5 +4224,6 @@ export async function archiveGiteaRepo(
console.error(`[Archive] Could not mark repository ${owner}/${repo} as archived:`, error);
console.log(`[Archive] Repository ${owner}/${repo} data is preserved but not marked as archived`);
// Don't throw - we want cleanup to continue for other repos
return { archivedName: null };
}
}
@@ -0,0 +1,52 @@
/**
* Unit tests for the pure orphan-verdict decision logic regression coverage
* for issue #331's root cause: `identifyOrphanedRepositories()` treated a DB
* repository as "orphaned" the moment it was missing from a single bulk
* GitHub fetch (owned+collaborator+org repos, plus starred repos). That bulk
* fetch can be transiently incomplete (rate-limit timing, GraphQL star-list
* pagination quirks, org-allowlist edge cases, etc.), producing false
* positives that got archived (renamed to `archived-{name}` in Gitea/Forgejo)
* even though the repo was never actually removed/unstarred on GitHub.
*
* The fix adds a second, targeted confirmation call for any repo that merely
* *looks* orphaned from the bulk list before finalizing it as such.
* `resolveOrphanVerdict` is the pure decision function extracted from that
* flow (similar in spirit to classifyAssetsForReconciliation /
* classifyReleasesForReconciliation in gitea-releases.test.ts) so the
* decision logic itself is unit-testable without hitting the DB or octokit.
*/
import { describe, test, expect } from "bun:test";
import { resolveOrphanVerdict } from "./repository-cleanup-service";
describe("resolveOrphanVerdict", () => {
test("repo present in the bulk fetch is never orphaned, regardless of the direct check", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: true, directCheckConfirmsGone: true })
).toBe(false);
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: true, directCheckConfirmsGone: false })
).toBe(false);
});
test("repo missing from the bulk fetch is orphaned only when the direct check confirms it's gone (404)", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: true })
).toBe(true);
});
test("repo missing from the bulk fetch but still found by the direct check is NOT orphaned (bulk fetch was incomplete)", () => {
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: false })
).toBe(false);
});
test("repo missing from the bulk fetch whose direct check itself failed (network error, rate limit, etc.) fails safe as NOT orphaned", () => {
// directCheckConfirmsGone is only true for a clean, explicit 404 — any
// other outcome (including a failed verification call) is represented
// as false by the caller, which must resolve to "not orphaned" here.
expect(
resolveOrphanVerdict({ fullNameFoundInBulkList: false, directCheckConfirmsGone: false })
).toBe(false);
});
});
+143 -9
View File
@@ -15,6 +15,37 @@ import { isMirrorableGitHubRepo } from '@/lib/repo-eligibility';
let cleanupInterval: NodeJS.Timeout | null = null;
let isCleanupRunning = false;
/**
* Decide whether a DB repository that appears to be missing from the bulk
* GitHub fetch should actually be treated as orphaned.
*
* The bulk fetch (owned+collaborator+org repos, plus starred repos) that
* feeds `fullNameFoundInBulkList` can be transiently incomplete rate-limit
* timing, GraphQL star-list pagination quirks, org-allowlist edge cases,
* etc. so a repo missing from it is only a *candidate*, not a confirmed
* orphan. It is only orphaned when a direct, targeted GitHub call ALSO
* confirms the repo is gone (a clean 404). Any other outcome the repo
* still exists, or the direct check itself failed for some other reason
* (network error, rate limit, 5xx, timeout) must NOT be treated as
* orphaned; this fails safe and matches the existing fail-safe philosophy
* already in this module for GitHub API errors.
*
* Pure/exported so the decision logic is unit-testable without hitting the
* DB or octokit (see repository-cleanup-service.test.ts).
*/
export function resolveOrphanVerdict({
fullNameFoundInBulkList,
directCheckConfirmsGone,
}: {
fullNameFoundInBulkList: boolean;
directCheckConfirmsGone: boolean;
}): boolean {
if (fullNameFoundInBulkList) {
return false;
}
return directCheckConfirmsGone;
}
/**
* Identify orphaned repositories for a user
* These are repositories that exist in our database (and likely in Gitea)
@@ -80,8 +111,12 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
.where(eq(repositories.userId, userId));
// Only identify repositories as orphaned if we successfully accessed GitHub
// This prevents false positives when GitHub is down or account is inaccessible
const orphanedRepos = dbRepos.filter(repo => {
// This prevents false positives when GitHub is down or account is inaccessible.
//
// First pass (sync, cheap): filter down to repos that merely *look*
// orphaned based on map membership against the single bulk fetch above.
// This is the false-positive-prone signal — see resolveOrphanVerdict.
const candidateOrphans = dbRepos.filter(repo => {
// Skip repositories we've already archived/preserved
if (repo.status === 'archived' || repo.isArchived) {
console.log(`[Repository Cleanup] Skipping ${repo.fullName} - already archived`);
@@ -97,6 +132,8 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
const githubRepo = githubReposByFullName.get(repo.fullName);
if (!githubRepo) {
// Missing from the bulk list — candidate for direct confirmation below,
// not yet a confirmed orphan.
return true;
}
@@ -107,11 +144,93 @@ async function identifyOrphanedRepositories(config: any): Promise<any[]> {
return false;
});
if (candidateOrphans.length === 0) {
return [];
}
// Second pass (async, targeted): confirm each candidate directly against
// GitHub before finalizing it as orphaned. This only adds extra API calls
// for the (presumably small) set of repos that look orphaned, not for
// every repo, so it shouldn't meaningfully increase rate-limit pressure
// in the common case (few or no orphans per run). Promise.allSettled so
// one repo's verification failure can't block the others.
const verificationOutcomes = await Promise.allSettled(
candidateOrphans.map(async (repo) => {
if (repo.isStarred) {
try {
await octokit.rest.activity.checkRepoIsStarredByAuthenticatedUser({
owner: repo.owner,
repo: repo.name,
});
// Resolves (no throw) => still starred; the bulk star fetch
// missed it. Fail safe: do not treat as orphaned.
return { repo, directCheckConfirmsGone: false };
} catch (starError: any) {
if (starError?.status === 404) {
return { repo, directCheckConfirmsGone: true };
}
console.warn(
`[Repository Cleanup] Direct star-check for ${repo.fullName} failed with a non-404 error; skipping this cycle to be safe: ${
starError instanceof Error ? starError.message : String(starError)
}`
);
return { repo, directCheckConfirmsGone: false };
}
}
try {
await octokit.rest.repos.get({ owner: repo.owner, repo: repo.name });
// Resolves (no throw) => repo still exists; the bulk fetch missed
// it (e.g. an org-allowlist edge case). Fail safe: not orphaned.
return { repo, directCheckConfirmsGone: false };
} catch (repoError: any) {
if (repoError?.status === 404) {
return { repo, directCheckConfirmsGone: true };
}
console.warn(
`[Repository Cleanup] Direct existence check for ${repo.fullName} failed with a non-404 error; skipping this cycle to be safe: ${
repoError instanceof Error ? repoError.message : String(repoError)
}`
);
return { repo, directCheckConfirmsGone: false };
}
})
);
const orphanedRepos = verificationOutcomes
.map((outcome, index) => {
if (outcome.status !== 'fulfilled') {
const repo = candidateOrphans[index];
console.warn(
`[Repository Cleanup] Direct orphan verification threw unexpectedly for ${repo.fullName}; skipping this cycle to be safe: ${
outcome.reason instanceof Error ? outcome.reason.message : String(outcome.reason)
}`
);
return null;
}
const { repo, directCheckConfirmsGone } = outcome.value;
const isOrphaned = resolveOrphanVerdict({
fullNameFoundInBulkList: false,
directCheckConfirmsGone,
});
if (!isOrphaned) {
return null;
}
console.log(
`[Repository Cleanup] Confirmed orphaned via direct GitHub check: ${repo.fullName}`
);
return repo;
})
.filter((repo): repo is (typeof dbRepos)[number] => repo !== null);
if (orphanedRepos.length > 0) {
console.log(`[Repository Cleanup] Found ${orphanedRepos.length} orphaned repositories for user ${userId}`);
}
return orphanedRepos;
} catch (error) {
console.error(`[Repository Cleanup] Error identifying orphaned repositories for user ${userId}:`, error);
@@ -185,15 +304,30 @@ async function handleOrphanedRepository(
// Non-fatal; continue with best guess
}
await archiveGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
// Update database status
await db.update(repositories).set({
const { archivedName } = await archiveGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
// Update database status. If the archive call renamed the repo in Gitea
// (mirror path), persist the new location so a subsequent "Manual Sync"
// (the UI's documented path for refreshing an archived mirror) can find
// it by its actual current name instead of the stale pre-rename one —
// otherwise syncGiteaRepoEnhanced looks up a name that no longer exists
// and Gitea returns HTTP 405 ("not a pull mirror").
//
// Only mirroredLocation gets the Gitea-side `archived-{name}` value.
// Do NOT write it into `name`: repositories.name is consumed as the
// GITHUB repo name elsewhere (release listing, force-push detection),
// and mirroredLocation alone is what syncGiteaRepoEnhanced resolves
// first when locating the Gitea mirror.
const dbUpdate: Record<string, any> = {
status: 'archived',
isArchived: true,
errorMessage: 'Repository archived - no longer in GitHub',
updatedAt: new Date(),
}).where(eq(repositories.id, repo.id));
};
if (archivedName && archivedName !== giteaRepoName) {
dbUpdate.mirroredLocation = `${giteaOwner}/${archivedName}`;
}
await db.update(repositories).set(dbUpdate).where(eq(repositories.id, repo.id));
// Create event
await publishEvent({