[2025-12-18 15:40:05,058][ INFO][PID:1560876] Marking build as starting [2025-12-18 15:40:05,122][ INFO][PID:1560876] Checking that /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/repodata/repomd.xml exists [2025-12-18 15:40:05,237][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:05,238][ INFO][PID:1560876] VM allocation process starts [2025-12-18 15:40:05,255][ INFO][PID:1560876] Trying to allocate VM: ResallocHost, ticket_id=425151, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-18 15:40:10,262][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:15,281][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:20,284][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:25,287][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:30,292][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:35,294][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:40,298][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:45,299][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:50,310][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:40:55,482][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:00,529][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:05,618][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:10,910][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:15,917][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:20,963][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:25,966][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:30,977][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:35,978][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:40,980][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:45,981][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:50,983][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:41:55,994][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:01,008][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:06,010][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:11,012][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:16,013][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:21,015][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:26,017][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:31,020][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:36,021][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:41,022][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:46,027][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:51,031][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:42:56,032][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:01,034][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:06,038][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:11,051][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:16,077][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:21,079][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:26,082][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:31,084][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:36,090][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:41,092][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:46,097][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:51,101][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:43:56,103][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:01,105][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:06,111][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:11,118][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:16,120][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:21,130][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:26,137][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:31,138][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:36,140][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:41,145][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:46,146][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:51,154][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:44:56,157][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:01,158][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:06,160][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:11,163][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:16,165][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:21,169][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:26,172][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:31,174][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:36,179][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:41,186][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:46,188][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:51,189][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:45:56,194][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:01,199][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:06,203][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:11,204][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:16,209][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:21,212][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:26,222][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:31,232][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:36,237][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:41,241][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:46,243][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:51,246][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:46:56,248][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:01,252][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:06,255][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:11,258][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:16,260][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:21,262][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:23,621][ INFO][PID:1560876] Allocated host ResallocHost, ticket_id=425151, hostname=54.167.43.141, name=aws_x86_64_spot_prod_00678557_20251218_154434, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-18 15:47:23,622][ INFO][PID:1560876] Allocating ssh connection to builder [2025-12-18 15:47:23,623][ INFO][PID:1560876] Checking that builder machine is OK [2025-12-18 15:47:23,996][ INFO][PID:1560876] Running remote command: copr-builder-ready fedora-41-x86_64 [2025-12-18 15:47:24,245][ INFO][PID:1560876] Red Hat subscription not needed for fedora-41-x86_64 Builder is ready to be used [2025-12-18 15:47:24,252][ INFO][PID:1560876] Filling build.info file with builder info [2025-12-18 15:47:24,256][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:24,259][ INFO][PID:1560876] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "nmstate", "project_name": "nmstate-git", "project_dirname": "nmstate-git", "submitter": "https://api.github.com/users/cathay4t", "ended_on": null, "started_on": 1766072844.2593193, "submitted_on": null, "status": 3, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9931235-fedora-41-x86_64", "build_id": 9931235, "package_name": "nmstate", "package_version": "2.2.58-0.20251218.2699git1ee9d54e", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/nmstate/nmstate-git/nmstate", "git_hash": "f14de406dab383b35186a3baa120480680468ddd", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "nmstate/nmstate-git--https://api.github.com/users/cathay4t", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/nmstate/nmstate-git/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/nmstate/nmstate-git", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/nmstate/nmstate-git", "result_dir": "09931235-nmstate", "built_packages": "", "id": 9931235, "mockchain_macros": { "copr_username": "nmstate", "copr_projectname": "nmstate-git", "vendor": "Fedora Project COPR (nmstate/nmstate-git)" } } ] } [2025-12-18 15:47:24,355][ INFO][PID:1560876] Sending fedora-messaging bus message in build.start [2025-12-18 15:47:25,055][ INFO][PID:1560876] Sending fedora-messaging bus message in chroot.start [2025-12-18 15:47:25,092][ INFO][PID:1560876] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9931235-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached [2025-12-18 15:47:25,431][ INFO][PID:1560876] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.167.43.141' (ED25519) to the list of known hosts. [2025-12-18 15:47:25,435][ INFO][PID:1560876] Downloading the builder-live.log file, attempt 1 [2025-12-18 15:47:25,436][ INFO][PID:1560876] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.167.43.141 copr-rpmbuild-log [2025-12-18 15:47:30,448][ INFO][PID:1560876] Periodic builder liveness probe: alive [2025-12-18 15:47:30,448][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:35,449][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:40,451][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:45,458][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:50,460][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:47:55,464][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:00,467][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:05,470][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:10,481][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:15,488][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:20,490][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:25,493][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:30,499][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:35,504][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:40,506][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:45,521][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:50,523][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:48:55,525][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:00,528][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:05,534][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:10,536][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:15,542][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:20,550][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:25,555][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:30,558][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:35,560][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:40,563][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:45,605][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:50,626][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:49:55,701][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:00,704][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:05,715][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:10,720][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:15,809][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:20,822][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:25,997][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:31,013][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:36,023][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:41,046][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:46,052][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:51,062][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:50:54,740][ ERROR][PID:1560876] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-18 15:50:56,080][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:01,091][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:04,742][WARNING][PID:1560876] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-18 15:51:06,176][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:11,182][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:14,743][ INFO][PID:1560876] Downloading the builder-live.log file, attempt 2 [2025-12-18 15:51:14,745][ INFO][PID:1560876] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.167.43.141 copr-rpmbuild-log [2025-12-18 15:51:16,199][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:21,201][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:26,390][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:31,458][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:36,462][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:41,523][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:46,811][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:51,819][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:51:56,832][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:01,842][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:06,844][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:11,847][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:14,813][ ERROR][PID:1560876] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-18 15:52:16,849][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:21,850][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:24,871][WARNING][PID:1560876] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-18 15:52:26,852][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:31,854][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:34,872][ INFO][PID:1560876] Downloading the builder-live.log file, attempt 3 [2025-12-18 15:52:34,874][ INFO][PID:1560876] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.167.43.141 copr-rpmbuild-log [2025-12-18 15:52:36,856][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:41,859][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:46,862][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:51,865][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:52:56,866][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:01,875][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:06,879][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:11,884][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:16,885][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:21,888][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:26,894][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:31,899][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:34,949][ ERROR][PID:1560876] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-18 15:53:36,902][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:41,908][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:44,956][WARNING][PID:1560876] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-18 15:53:46,911][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:51,912][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:53:54,961][ INFO][PID:1560876] Downloading the builder-live.log file, attempt 4 [2025-12-18 15:53:54,968][ INFO][PID:1560876] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.167.43.141 copr-rpmbuild-log [2025-12-18 15:53:56,920][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:01,936][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:06,955][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:11,971][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:16,989][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:22,001][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:27,011][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:32,013][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:37,029][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:42,132][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:47,158][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:52,213][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:54:55,079][ ERROR][PID:1560876] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-18 15:54:57,227][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:02,232][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:05,089][ ERROR][PID:1560876] Giving up for unstable SSH, failures: 1766073064.7420607, 1766073144.8715508, 1766073224.9568434, 1766073305.0894382 [2025-12-18 15:55:05,091][ ERROR][PID:1560876] Re-try request for task on 'ResallocHost, ticket_id=425151, hostname=54.167.43.141, name=aws_x86_64_spot_prod_00678557_20251218_154434, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-18 15:55:05,092][ INFO][PID:1560876] Releasing VM back to pool [2025-12-18 15:55:05,106][ INFO][PID:1560876] Retry #1 (on other host) [2025-12-18 15:55:05,106][ INFO][PID:1560876] Checking that /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/repodata/repomd.xml exists [2025-12-18 15:55:05,107][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:05,108][ INFO][PID:1560876] VM allocation process starts [2025-12-18 15:55:05,123][ INFO][PID:1560876] Trying to allocate VM: ResallocHost, ticket_id=425669, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-18 15:55:10,125][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:15,130][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:20,134][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:23,218][ INFO][PID:1560876] Allocated host ResallocHost, ticket_id=425669, hostname=98.92.253.2, name=aws_x86_64_normalreserved_prod_00678493_20251218_154300, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-18 15:55:23,219][ INFO][PID:1560876] Allocating ssh connection to builder [2025-12-18 15:55:23,220][ INFO][PID:1560876] Checking that builder machine is OK [2025-12-18 15:55:23,592][ INFO][PID:1560876] Running remote command: copr-builder-ready fedora-41-x86_64 [2025-12-18 15:55:23,812][ INFO][PID:1560876] Red Hat subscription not needed for fedora-41-x86_64 Builder is ready to be used [2025-12-18 15:55:23,814][ INFO][PID:1560876] Filling build.info file with builder info [2025-12-18 15:55:23,928][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:23,929][ INFO][PID:1560876] Not re-notifying FE and msg buses for the new host. [2025-12-18 15:55:23,930][ INFO][PID:1560876] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9931235-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached [2025-12-18 15:55:24,290][ INFO][PID:1560876] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '98.92.253.2' (ED25519) to the list of known hosts. [2025-12-18 15:55:24,291][ INFO][PID:1560876] Downloading the builder-live.log file, attempt 1 [2025-12-18 15:55:24,292][ INFO][PID:1560876] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.253.2 copr-rpmbuild-log [2025-12-18 15:55:29,292][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:34,295][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:39,300][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:44,303][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:49,305][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:54,308][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:55:59,312][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:04,375][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:09,378][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:14,385][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:19,387][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:24,390][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:29,392][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:34,393][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:39,504][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:44,509][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:49,511][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:54,512][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:56:59,515][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:04,516][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:09,518][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:14,523][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:19,524][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:24,525][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:29,539][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:34,560][ INFO][PID:1560876] Periodic builder liveness probe: alive [2025-12-18 15:57:34,561][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:39,565][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:44,567][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:49,569][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:54,570][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:57:59,572][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:04,667][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:09,670][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:14,672][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:19,675][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:24,678][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:29,681][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:34,683][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:39,684][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:44,686][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:49,706][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:54,725][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:58:59,735][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:04,838][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:09,884][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:14,894][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:19,983][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:24,985][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:30,027][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:35,061][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:40,096][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:45,107][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:50,123][ INFO][PID:1560876] Checking for cancel request [2025-12-18 15:59:55,161][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:00,173][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:05,177][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:10,181][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:15,182][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:20,184][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:25,528][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:30,533][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:35,542][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:40,544][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:45,545][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:50,551][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:00:55,553][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:00,556][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:05,563][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:10,566][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:15,576][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:20,577][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:25,580][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:30,581][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:35,589][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:40,733][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:45,734][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:50,747][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:01:55,751][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:00,753][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:05,757][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:10,759][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:15,763][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:20,764][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:25,767][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:30,787][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:35,793][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:40,794][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:45,799][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:50,801][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:02:55,802][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:00,805][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:05,815][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:10,817][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:15,818][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:20,820][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:25,828][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:30,831][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:35,855][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:40,856][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:45,862][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:50,868][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:03:55,870][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:04:00,876][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:04:05,914][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:04:10,945][ INFO][PID:1560876] Checking for cancel request [2025-12-18 16:04:11,421][ INFO][PID:1560876] Downloading results from builder [2025-12-18 16:04:11,422][ INFO][PID:1560876] rsyncing of mockbuilder@98.92.253.2:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate started [2025-12-18 16:04:11,424][ INFO][PID:1560876] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@98.92.253.2:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/ &> /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/build-09931235.rsync.log [2025-12-18 16:04:12,399][ INFO][PID:1560876] rsyncing finished. [2025-12-18 16:04:12,400][ INFO][PID:1560876] Releasing VM back to pool [2025-12-18 16:04:12,411][ INFO][PID:1560876] Searching for 'success' file in resultdir [2025-12-18 16:04:12,413][ INFO][PID:1560876] Going to sign pkgs from source: 9931235-fedora-41-x86_64 in chroot: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64 [2025-12-18 16:04:12,414][ INFO][PID:1560876] Calling '/bin/sign -u nmstate#nmstate-git@copr.fedorahosted.org -p' (attempt #1) [2025-12-18 16:04:12,681][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-static-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm' (attempt #1) [2025-12-18 16:04:13,594][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-static-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm [2025-12-18 16:04:13,595][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/python3-libnmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm' (attempt #1) [2025-12-18 16:04:14,317][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/python3-libnmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm [2025-12-18 16:04:14,318][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-devel-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm' (attempt #1) [2025-12-18 16:04:14,922][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-devel-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm [2025-12-18 16:04:14,923][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-libs-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm' (attempt #1) [2025-12-18 16:04:15,875][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-libs-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm [2025-12-18 16:04:15,881][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.src.rpm' (attempt #1) [2025-12-18 16:04:17,377][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.src.rpm [2025-12-18 16:04:17,378][ INFO][PID:1560876] Calling '/bin/sign -4 -h sha256 -u nmstate#nmstate-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm' (attempt #1) [2025-12-18 16:04:18,220][ INFO][PID:1560876] signed rpm: /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/nmstate-2.2.58-0.20251218.2699git1ee9d54e.fc41.x86_64.rpm [2025-12-18 16:04:18,221][ INFO][PID:1560876] Sign done [2025-12-18 16:04:18,225][ INFO][PID:1560876] Incremental createrepo run, adding 09931235-nmstate into https://download.copr.fedorainfracloud.org/results/nmstate/nmstate-git/fedora-41-x86_64, (auto-create-repo=True) [2025-12-18 16:04:18,229][ INFO][PID:1560876] Running command 'copr-repo --batched /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64 --add 09931235-nmstate --no-appstream-metadata' as PID 1824416 [2025-12-18 16:04:20,053][ INFO][PID:1560876] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64 --add 09931235-nmstate --no-appstream-metadata) [2025-12-18 16:04:20,056][ INFO][PID:1560876] Getting build details [2025-12-18 16:04:20,058][ INFO][PID:1560876] Listing built binary packages in /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate [2025-12-18 16:04:20,061][ INFO][PID:1560876] Built packages: ['nmstate-static 2.2.58', 'nmstate 2.2.58', 'nmstate-devel 2.2.58', 'nmstate-libs 2.2.58', 'python3-libnmstate 2.2.58'] [2025-12-18 16:04:20,064][ INFO][PID:1560876] build details: {'built_packages': 'nmstate-static 2.2.58\nnmstate 2.2.58\nnmstate-devel 2.2.58\nnmstate-libs 2.2.58\npython3-libnmstate 2.2.58'} [2025-12-18 16:04:20,067][ INFO][PID:1560876] Retrieving pubkey [2025-12-18 16:04:20,070][ INFO][PID:1560876] Calling '/bin/sign -u nmstate#nmstate-git@copr.fedorahosted.org -p' (attempt #1) [2025-12-18 16:04:20,269][ INFO][PID:1560876] Added pubkey for user nmstate project nmstate-git into: /var/lib/copr/public_html/results/nmstate/nmstate-git/pubkey.gpg [2025-12-18 16:04:20,270][ INFO][PID:1560876] Finished build: id=9931235 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/nmstate/nmstate-git chroot=fedora-41-x86_64 [2025-12-18 16:04:20,292][ INFO][PID:1560876] Worker succeeded build, took 536.3627967834473 [2025-12-18 16:04:20,299][ INFO][PID:1560876] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "nmstate", "project_name": "nmstate-git", "project_dirname": "nmstate-git", "submitter": "https://api.github.com/users/cathay4t", "ended_on": 1766073860.29209, "started_on": 1766073323.9292932, "submitted_on": null, "status": 1, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9931235-fedora-41-x86_64", "build_id": 9931235, "package_name": "nmstate", "package_version": "2.2.58-0.20251218.2699git1ee9d54e", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/nmstate/nmstate-git/nmstate", "git_hash": "f14de406dab383b35186a3baa120480680468ddd", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "nmstate/nmstate-git--https://api.github.com/users/cathay4t", "results": { "packages": [ { "name": "nmstate-static", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "x86_64" }, { "name": "nmstate", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "x86_64" }, { "name": "nmstate-devel", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "x86_64" }, { "name": "nmstate", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "src" }, { "name": "nmstate-libs", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "x86_64" }, { "name": "python3-libnmstate", "epoch": null, "version": "2.2.58", "release": "0.20251218.2699git1ee9d54e.fc41", "arch": "x86_64" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/nmstate/nmstate-git/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/nmstate/nmstate-git", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/nmstate/nmstate-git", "result_dir": "09931235-nmstate", "built_packages": "nmstate-static 2.2.58\nnmstate 2.2.58\nnmstate-devel 2.2.58\nnmstate-libs 2.2.58\npython3-libnmstate 2.2.58", "id": 9931235, "mockchain_macros": { "copr_username": "nmstate", "copr_projectname": "nmstate-git", "vendor": "Fedora Project COPR (nmstate/nmstate-git)" } } ] } [2025-12-18 16:04:20,471][ INFO][PID:1560876] Sending fedora-messaging bus message in build.end [2025-12-18 16:04:20,561][ INFO][PID:1560876] Compressing /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/builder-live.log by gzip [2025-12-18 16:04:20,567][ INFO][PID:1560876] Running command 'gzip /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/builder-live.log' as PID 1824802 [2025-12-18 16:04:20,715][ INFO][PID:1560876] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/builder-live.log) [2025-12-18 16:04:20,716][ INFO][PID:1560876] Compressing /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/backend.log by gzip [2025-12-18 16:04:20,719][ INFO][PID:1560876] Running command 'gzip /var/lib/copr/public_html/results/nmstate/nmstate-git/fedora-41-x86_64/09931235-nmstate/backend.log' as PID 1824815