[2025-12-10 20:20:53,521][ INFO][PID:920043] Marking build as starting [2025-12-10 20:20:53,639][ INFO][PID:920043] Checking that /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/repodata/repomd.xml exists [2025-12-10 20:20:53,640][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:20:53,642][ INFO][PID:920043] VM allocation process starts [2025-12-10 20:20:53,693][ INFO][PID:920043] Trying to allocate VM: ResallocHost, ticket_id=267898, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-10 20:20:58,699][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:03,708][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:05,793][ INFO][PID:920043] Allocated host ResallocHost, ticket_id=267898, hostname=54.173.72.121, name=aws_x86_64_spot_prod_00329474_20251210_201929, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-10 20:21:05,794][ INFO][PID:920043] Allocating ssh connection to builder [2025-12-10 20:21:05,795][ INFO][PID:920043] Checking that builder machine is OK [2025-12-10 20:21:06,142][ INFO][PID:920043] Running remote command: copr-builder-ready centos-stream-9-x86_64 [2025-12-10 20:21:06,356][ INFO][PID:920043] Red Hat subscription not needed for centos-stream-9-x86_64 Builder is ready to be used [2025-12-10 20:21:06,357][ INFO][PID:920043] Filling build.info file with builder info [2025-12-10 20:21:06,357][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:06,358][ INFO][PID:920043] Sending build state back to frontend: { "builds": [ { "timeout": 90000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@fedora-llvm-team", "project_name": "clang-fedora-centos-testing", "project_dirname": "clang-fedora-centos-testing", "submitter": "tstellar", "ended_on": null, "started_on": 1765398066.3582716, "submitted_on": null, "status": 3, "chroot": "centos-stream-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9896313-centos-stream-9-x86_64", "build_id": 9896313, "package_name": "dotnet10.0", "package_version": "10.0.100-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/clang-fedora-centos-testing/dotnet10.0", "git_hash": "b4fcfc51f0f9be70227609e3d74384f4daa0725c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@fedora-llvm-team/clang-fedora-centos-testing--tstellar", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": 1000 }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-compat-packages/centos-stream-9-x86_64/", "id": "copr_fedora_llvm_team_llvm_compat_packages", "name": "Additional repo copr_fedora_llvm_team_llvm_compat_packages" }, { "baseurl": "https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/x86_64/", "id": "https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_x86_64", "name": "Additional repo https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_x86_64" }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20251210/centos-stream-9-x86_64/", "id": "copr_fedora_llvm_team_llvm_snapshots_big_merge_20251210", "module_hotfixes": true, "name": "Additional repo copr_fedora_llvm_team_llvm_snapshots_big_merge_20251210" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": 1000, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-fedora-centos-testing", "result_dir": "09896313-dotnet10.0", "built_packages": "", "id": 9896313, "mockchain_macros": { "copr_username": "@fedora-llvm-team", "copr_projectname": "clang-fedora-centos-testing", "vendor": "Fedora Project COPR (@fedora-llvm-team/clang-fedora-centos-testing)" } } ] } [2025-12-10 20:21:06,412][ INFO][PID:920043] Sending fedora-messaging bus message in build.start [2025-12-10 20:21:07,145][ INFO][PID:920043] Sending fedora-messaging bus message in chroot.start [2025-12-10 20:21:07,178][ INFO][PID:920043] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9896313-centos-stream-9-x86_64 --chroot centos-stream-9-x86_64 --detached [2025-12-10 20:21:07,511][ INFO][PID:920043] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.173.72.121' (ED25519) to the list of known hosts. [2025-12-10 20:21:07,512][ INFO][PID:920043] Downloading the builder-live.log file, attempt 1 [2025-12-10 20:21:07,516][ INFO][PID:920043] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.72.121 copr-rpmbuild-log [2025-12-10 20:21:12,539][ INFO][PID:920043] Periodic builder liveness probe: alive [2025-12-10 20:21:12,540][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:17,546][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:22,551][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:27,555][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:32,557][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:37,560][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:42,561][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:47,563][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:52,566][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:21:57,571][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:02,572][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:07,574][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:12,576][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:17,579][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:22,583][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:27,587][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:32,588][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:37,590][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:42,592][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:47,594][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:52,595][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:22:57,596][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:02,597][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:07,599][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:12,600][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:17,604][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:22,605][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:27,606][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:32,610][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:37,611][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:42,613][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:47,614][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:52,615][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:23:57,616][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:02,617][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:07,619][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:12,620][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:17,622][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:22,623][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:27,624][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:32,625][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:37,626][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:42,627][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:47,628][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:52,629][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:24:57,631][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:02,632][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:07,633][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:12,634][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:17,635][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:22,636][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:27,638][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:32,639][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:37,640][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:42,641][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:47,642][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:52,643][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:25:57,645][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:02,646][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:07,647][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:12,649][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:17,650][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:22,651][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:27,652][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:32,561][ ERROR][PID:920043] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-10 20:26:32,654][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:37,655][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:42,564][WARNING][PID:920043] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-10 20:26:42,657][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:47,664][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:52,565][ INFO][PID:920043] Downloading the builder-live.log file, attempt 2 [2025-12-10 20:26:52,567][ INFO][PID:920043] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.72.121 copr-rpmbuild-log [2025-12-10 20:26:52,665][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:26:57,666][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:02,670][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:07,677][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:12,679][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:17,680][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:22,681][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:27,682][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:32,683][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:37,684][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:42,685][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:47,686][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:52,634][ ERROR][PID:920043] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-10 20:27:52,687][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:27:57,688][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:02,636][WARNING][PID:920043] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-10 20:28:02,689][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:07,690][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:12,637][ INFO][PID:920043] Downloading the builder-live.log file, attempt 3 [2025-12-10 20:28:12,638][ INFO][PID:920043] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.72.121 copr-rpmbuild-log [2025-12-10 20:28:12,691][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:17,692][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:22,693][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:27,699][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:32,700][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:37,701][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:42,702][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:47,704][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:52,706][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:28:57,707][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:02,708][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:07,710][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:12,705][ ERROR][PID:920043] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-10 20:29:12,711][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:17,713][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:22,715][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:22,810][WARNING][PID:920043] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-10 20:29:27,716][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:32,717][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:32,811][ INFO][PID:920043] Downloading the builder-live.log file, attempt 4 [2025-12-10 20:29:32,812][ INFO][PID:920043] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.72.121 copr-rpmbuild-log [2025-12-10 20:29:37,718][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:42,719][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:47,720][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:52,721][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:29:57,722][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:02,723][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:07,725][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:12,727][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:17,729][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:22,732][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:28,935][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:33,056][ ERROR][PID:920043] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-10 20:30:35,286][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:41,241][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:43,057][ ERROR][PID:920043] Giving up for unstable SSH, failures: 1765398402.5647562, 1765398482.636581, 1765398562.8106792, 1765398643.0571427 [2025-12-10 20:30:43,058][ ERROR][PID:920043] Re-try request for task on 'ResallocHost, ticket_id=267898, hostname=54.173.72.121, name=aws_x86_64_spot_prod_00329474_20251210_201929, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-10 20:30:43,058][ INFO][PID:920043] Releasing VM back to pool [2025-12-10 20:30:43,069][ INFO][PID:920043] Retry #1 (on other host) [2025-12-10 20:30:43,069][ INFO][PID:920043] Checking that /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/repodata/repomd.xml exists [2025-12-10 20:30:43,070][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:43,070][ INFO][PID:920043] VM allocation process starts [2025-12-10 20:30:43,082][ INFO][PID:920043] Trying to allocate VM: ResallocHost, ticket_id=268125, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-10 20:30:46,096][ INFO][PID:920043] Allocated host ResallocHost, ticket_id=268125, hostname=18.212.250.99, name=aws_x86_64_normalreserved_prod_00329564_20251210_202057, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-10 20:30:46,097][ INFO][PID:920043] Allocating ssh connection to builder [2025-12-10 20:30:46,098][ INFO][PID:920043] Checking that builder machine is OK [2025-12-10 20:30:46,433][ INFO][PID:920043] Running remote command: copr-builder-ready centos-stream-9-x86_64 [2025-12-10 20:30:47,376][ INFO][PID:920043] Red Hat subscription not needed for centos-stream-9-x86_64 Builder is ready to be used [2025-12-10 20:30:47,376][ INFO][PID:920043] Filling build.info file with builder info [2025-12-10 20:30:47,377][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:47,378][ INFO][PID:920043] Not re-notifying FE and msg buses for the new host. [2025-12-10 20:30:47,378][ INFO][PID:920043] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9896313-centos-stream-9-x86_64 --chroot centos-stream-9-x86_64 --detached [2025-12-10 20:30:47,725][ INFO][PID:920043] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '18.212.250.99' (ED25519) to the list of known hosts. [2025-12-10 20:30:47,726][ INFO][PID:920043] Downloading the builder-live.log file, attempt 1 [2025-12-10 20:30:47,728][ INFO][PID:920043] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.212.250.99 copr-rpmbuild-log [2025-12-10 20:30:52,727][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:30:58,888][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:06,227][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:13,184][ INFO][PID:920043] Periodic builder liveness probe: alive [2025-12-10 20:31:13,185][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:18,185][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:23,317][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:29,007][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:35,543][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:40,590][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:45,594][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:50,596][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:31:55,598][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:00,620][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:05,713][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:10,717][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:15,720][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:20,724][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:25,726][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:30,727][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:35,728][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:40,729][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:45,733][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:50,735][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:32:55,736][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:00,737][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:05,738][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:10,739][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:15,741][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:20,742][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:25,754][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:30,763][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:35,766][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:40,776][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:45,782][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:50,795][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:33:56,193][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:01,365][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:06,436][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:11,437][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:16,439][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:21,464][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:26,466][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:31,981][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:37,051][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:42,058][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:47,059][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:52,061][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:34:57,220][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:02,223][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:07,347][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:12,349][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:17,351][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:22,506][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:27,719][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:32,719][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:39,798][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:44,799][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:35:52,710][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:04,366][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:13,735][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:25,277][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:30,278][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:38,190][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:43,191][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:48,281][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:36:54,334][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:37:00,166][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:37:31,374][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:37:41,574][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:37:48,446][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:37:58,022][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:10,926][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:19,254][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:28,830][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:37,511][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:45,214][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:38:53,542][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:00,622][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:07,910][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:20,190][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:28,518][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:33,519][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:38,520][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:46,222][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:39:56,006][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:04,406][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:09,408][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:17,318][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:24,191][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:29,191][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:34,192][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:44,367][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:53,318][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:40:58,319][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:04,566][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:12,270][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:18,945][ INFO][PID:920043] Periodic builder liveness probe: alive [2025-12-10 20:41:21,437][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:26,438][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:33,311][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:38,311][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:43,312][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:53,094][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:41:58,095][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:03,096][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:08,097][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:14,551][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:19,552][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:24,553][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:29,554][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:34,555][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:39,556][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:44,557][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:49,558][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:54,559][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:42:59,882][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:04,883][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:09,884][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:14,886][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:19,887][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:24,888][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:29,889][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:34,891][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:39,892][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:44,893][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:49,894][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:54,895][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:43:59,896][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:04,898][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:09,899][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:14,900][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:19,901][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:24,902][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:29,904][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:34,905][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:39,906][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:44,907][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:49,909][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:54,911][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:44:59,913][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:04,914][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:09,915][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:14,920][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:19,921][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:24,922][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:29,923][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:34,924][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:39,925][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:45,029][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:50,030][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:45:55,031][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:00,032][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:05,034][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:10,037][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:15,038][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:20,039][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:25,040][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:30,041][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:35,042][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:40,043][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:45,044][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:50,045][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:46:55,046][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:00,047][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:05,048][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:10,049][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:15,050][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:20,051][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:25,052][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:30,053][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:35,054][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:40,055][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:45,056][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:50,057][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:47:55,058][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:00,059][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:05,060][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:10,061][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:15,062][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:20,064][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:25,065][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:30,065][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:35,066][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:40,067][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:45,068][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:50,070][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:48:55,072][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:00,081][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:05,082][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:10,145][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:15,163][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:20,170][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:25,172][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:30,173][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:35,341][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:40,344][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:45,346][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:50,348][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:49:55,350][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:00,351][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:05,352][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:10,353][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:15,354][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:20,356][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:25,358][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:30,360][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:35,362][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:40,380][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:45,426][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:50,456][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:50:55,757][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:00,768][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:05,772][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:10,774][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:15,775][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:20,776][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:25,785][ INFO][PID:920043] Periodic builder liveness probe: alive [2025-12-10 20:51:25,786][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:30,787][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:35,946][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:40,949][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:45,951][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:50,952][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:51:55,953][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:00,954][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:05,955][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:10,956][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:15,957][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:20,958][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:25,959][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:30,961][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:35,962][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:40,963][ INFO][PID:920043] Checking for cancel request [2025-12-10 20:52:44,565][ INFO][PID:920043] Downloading results from builder [2025-12-10 20:52:44,566][ INFO][PID:920043] rsyncing of mockbuilder@18.212.250.99:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0 started [2025-12-10 20:52:44,567][ INFO][PID:920043] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@18.212.250.99:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/ &> /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/build-09896313.rsync.log [2025-12-10 20:52:47,501][ INFO][PID:920043] rsyncing finished. [2025-12-10 20:52:47,502][ INFO][PID:920043] Releasing VM back to pool [2025-12-10 20:52:47,511][ INFO][PID:920043] Searching for 'success' file in resultdir [2025-12-10 20:52:47,512][ ERROR][PID:920043] Build failed: Backend process error: No success file => build failure [2025-12-10 20:52:47,513][ INFO][PID:920043] Finished build: id=9896313 failed=True timeout=90000 destdir=/var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing chroot=centos-stream-9-x86_64 [2025-12-10 20:52:47,514][ ERROR][PID:920043] Backend process error: No success file => build failure [2025-12-10 20:52:47,514][ INFO][PID:920043] Worker failed build, took 1320.136730670929 [2025-12-10 20:52:47,515][ INFO][PID:920043] Sending build state back to frontend: { "builds": [ { "timeout": 90000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@fedora-llvm-team", "project_name": "clang-fedora-centos-testing", "project_dirname": "clang-fedora-centos-testing", "submitter": "tstellar", "ended_on": 1765399967.514889, "started_on": 1765398647.3781583, "submitted_on": null, "status": 0, "chroot": "centos-stream-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9896313-centos-stream-9-x86_64", "build_id": 9896313, "package_name": "dotnet10.0", "package_version": "10.0.100-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/clang-fedora-centos-testing/dotnet10.0", "git_hash": "b4fcfc51f0f9be70227609e3d74384f4daa0725c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@fedora-llvm-team/clang-fedora-centos-testing--tstellar", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": 1000 }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-compat-packages/centos-stream-9-x86_64/", "id": "copr_fedora_llvm_team_llvm_compat_packages", "name": "Additional repo copr_fedora_llvm_team_llvm_compat_packages" }, { "baseurl": "https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/x86_64/", "id": "https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_x86_64", "name": "Additional repo https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_x86_64" }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20251210/centos-stream-9-x86_64/", "id": "copr_fedora_llvm_team_llvm_snapshots_big_merge_20251210", "module_hotfixes": true, "name": "Additional repo copr_fedora_llvm_team_llvm_snapshots_big_merge_20251210" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": 1000, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-fedora-centos-testing", "result_dir": "09896313-dotnet10.0", "built_packages": "", "id": 9896313, "mockchain_macros": { "copr_username": "@fedora-llvm-team", "copr_projectname": "clang-fedora-centos-testing", "vendor": "Fedora Project COPR (@fedora-llvm-team/clang-fedora-centos-testing)" } } ] } [2025-12-10 20:52:47,548][ INFO][PID:920043] Sending fedora-messaging bus message in build.end [2025-12-10 20:52:47,576][ INFO][PID:920043] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/builder-live.log by gzip [2025-12-10 20:52:47,577][ INFO][PID:920043] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/builder-live.log' as PID 1076719 [2025-12-10 20:52:48,381][ INFO][PID:920043] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/builder-live.log) [2025-12-10 20:52:48,382][ INFO][PID:920043] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/backend.log by gzip [2025-12-10 20:52:48,383][ INFO][PID:920043] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-fedora-centos-testing/centos-stream-9-x86_64/09896313-dotnet10.0/backend.log' as PID 1076792