[2026-01-13 23:31:50,641][ INFO][PID:2379347] Marking build as starting [2026-01-13 23:31:50,722][ INFO][PID:2379347] Checking that /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2026-01-13 23:31:50,723][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:31:50,724][ INFO][PID:2379347] VM allocation process starts [2026-01-13 23:31:50,774][ INFO][PID:2379347] Trying to allocate VM: ResallocHost, ticket_id=801308, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 23:31:53,819][ INFO][PID:2379347] Allocated host ResallocHost, ticket_id=801308, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01214895_20260113_232711, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 23:31:53,820][ INFO][PID:2379347] Allocating ssh connection to builder [2026-01-13 23:31:53,820][ INFO][PID:2379347] Checking that builder machine is OK [2026-01-13 23:31:54,669][ INFO][PID:2379347] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-13 23:31:55,170][ INFO][PID:2379347] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-13 23:31:55,170][ INFO][PID:2379347] Filling build.info file with builder info [2026-01-13 23:31:55,171][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:31:55,172][ INFO][PID:2379347] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "ryanabx", "project_name": "cosmic-epoch-tagged", "project_dirname": "cosmic-epoch-tagged", "submitter": "ryanabx", "ended_on": null, "started_on": 1768347115.1720536, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10005412-fedora-rawhide-ppc64le", "build_id": 10005412, "package_name": "cosmic-term", "package_version": "1.0.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ryanabx/cosmic-epoch-tagged/cosmic-term", "git_hash": "d1c5c6c97f8d08fa1e559b51cb5505375918f82a", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ryanabx/cosmic-epoch-tagged--ryanabx", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": true, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged", "result_dir": "10005412-cosmic-term", "built_packages": "", "id": 10005412, "mockchain_macros": { "copr_username": "ryanabx", "copr_projectname": "cosmic-epoch-tagged", "vendor": "Fedora Project COPR (ryanabx/cosmic-epoch-tagged)" } } ] } [2026-01-13 23:31:55,212][ INFO][PID:2379347] Sending fedora-messaging bus message in build.start [2026-01-13 23:31:55,718][ INFO][PID:2379347] Sending fedora-messaging bus message in chroot.start [2026-01-13 23:31:55,744][ INFO][PID:2379347] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10005412-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-13 23:31:56,763][ INFO][PID:2379347] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18d' (ED25519) to the list of known hosts. [2026-01-13 23:31:57,325][ INFO][PID:2379347] Downloading the builder-live.log file, attempt 1 [2026-01-13 23:31:57,364][ INFO][PID:2379347] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log [2026-01-13 23:32:02,354][ INFO][PID:2379347] Periodic builder liveness probe: alive [2026-01-13 23:32:02,355][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:07,356][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:12,357][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:17,358][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:22,359][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:27,360][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:32,361][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:37,362][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:42,363][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:47,364][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:52,365][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:32:57,368][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:02,369][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:07,370][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:12,371][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:17,372][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:22,373][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:27,374][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:32,375][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:37,376][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:42,377][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:47,378][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:52,379][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:33:57,380][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:02,382][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:07,383][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:12,384][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:17,384][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:22,385][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:27,386][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:32,387][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:37,388][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:42,427][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:47,428][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:52,429][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:34:57,430][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:02,432][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:07,433][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:12,434][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:17,435][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:22,436][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:27,437][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:32,438][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:37,439][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:42,440][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:47,441][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:52,442][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:35:57,443][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:02,444][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:07,445][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:12,446][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:17,447][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:22,448][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:27,449][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:32,450][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:37,451][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:42,452][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:47,453][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:52,454][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:36:57,455][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:02,456][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:07,457][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:12,366][ ERROR][PID:2379347] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-13 23:37:12,458][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:17,459][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:22,366][WARNING][PID:2379347] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-13 23:37:22,460][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:27,461][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:32,367][ INFO][PID:2379347] Downloading the builder-live.log file, attempt 2 [2026-01-13 23:37:32,369][ INFO][PID:2379347] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log [2026-01-13 23:37:32,483][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:37,484][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:42,485][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:47,486][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:52,487][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:37:57,488][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:02,489][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:07,490][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:12,491][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:17,492][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:22,493][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:27,494][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:32,494][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:37,495][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:42,496][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:47,497][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:52,498][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:38:57,499][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:02,500][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:07,501][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:12,502][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:17,503][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:22,504][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:27,505][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:32,506][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:37,507][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:42,508][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:47,509][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:52,510][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:39:57,511][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:02,511][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:07,512][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:12,513][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:17,514][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:22,515][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:27,516][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:32,517][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:37,518][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:42,519][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:47,520][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:52,521][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:40:57,522][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:02,523][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:07,523][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:12,524][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:17,525][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:22,526][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:27,527][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:32,528][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:37,529][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:42,530][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:47,531][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:52,532][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:41:57,532][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:02,539][ INFO][PID:2379347] Periodic builder liveness probe: alive [2026-01-13 23:42:02,539][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:07,540][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:12,541][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:17,542][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:22,543][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:27,544][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:32,545][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:37,546][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:42,547][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:47,548][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:52,549][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:42:57,550][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:02,551][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:07,552][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:12,553][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:17,554][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:22,555][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:27,556][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:32,557][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:37,558][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:42,558][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:47,559][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:52,560][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:43:57,561][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:02,562][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:07,563][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:12,564][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:17,565][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:22,566][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:27,567][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:32,568][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:37,569][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:42,570][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:47,571][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:52,572][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:44:57,573][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:02,574][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:07,576][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:12,577][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:17,578][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:22,654][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:27,655][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:32,656][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:37,657][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:42,658][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:47,669][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:52,714][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:45:57,715][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:02,764][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:07,769][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:12,771][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:17,773][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:22,773][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:27,827][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:32,833][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:37,837][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:42,842][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:47,886][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:52,901][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:46:57,902][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:02,903][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:08,054][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:13,055][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:18,057][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:23,058][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:28,060][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:33,061][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:38,062][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:43,064][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:48,065][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:53,066][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:47:58,067][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:03,068][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:08,139][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:13,238][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:18,239][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:23,240][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:28,241][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:33,243][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:38,244][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:43,246][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:48,247][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:53,248][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:48:58,248][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:03,249][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:08,250][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:13,251][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:18,262][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:23,263][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:28,264][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:33,312][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:38,313][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:43,314][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:48,315][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:53,316][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:49:58,317][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:03,318][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:08,319][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:13,633][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:18,635][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:23,638][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:28,639][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:33,640][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:38,641][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:43,642][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:48,642][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:53,643][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:50:58,814][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:03,815][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:08,816][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:13,817][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:18,818][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:23,818][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:28,820][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:33,821][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:38,822][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:43,823][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:48,991][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:53,991][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:51:58,993][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:03,999][ INFO][PID:2379347] Periodic builder liveness probe: alive [2026-01-13 23:52:04,000][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:09,001][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:14,001][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:19,002][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:24,003][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:29,011][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:34,032][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:39,117][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:44,215][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:49,218][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:54,225][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:52:59,337][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:04,338][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:09,339][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:14,340][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:19,341][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:24,533][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:29,534][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:34,535][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:39,536][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:44,537][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:49,538][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:54,572][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:53:59,574][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:04,576][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:09,577][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:14,616][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:19,617][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:24,618][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:29,620][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:34,620][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:39,621][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:44,622][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:49,623][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:54,624][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:54:59,625][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:04,626][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:09,626][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:14,627][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:19,628][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:24,653][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:29,660][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:34,661][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:39,662][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:44,801][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:49,803][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:54,804][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:55:59,805][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:04,835][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:09,836][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:14,837][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:19,838][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:24,879][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:29,880][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:34,901][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:39,902][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:45,039][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:50,040][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:56:55,345][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:00,346][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:05,393][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:10,395][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:15,396][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:20,398][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:25,400][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:30,401][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:35,750][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:40,752][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:45,753][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:50,754][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:57:55,755][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:00,756][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:05,929][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:10,931][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:15,932][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:21,013][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:26,015][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:31,143][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:36,146][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:41,148][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:46,208][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:51,210][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:58:56,212][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:01,213][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:06,215][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:11,235][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:16,236][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:21,326][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:26,326][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:31,453][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:36,454][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:41,454][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:46,455][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:51,456][ INFO][PID:2379347] Checking for cancel request [2026-01-13 23:59:56,484][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:01,485][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:06,859][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:11,860][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:16,861][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:21,862][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:26,863][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:31,864][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:36,865][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:41,866][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:46,867][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:51,868][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:00:53,261][ ERROR][PID:2379347] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-14 00:00:56,884][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:01,885][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:03,262][WARNING][PID:2379347] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-14 00:01:06,886][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:12,133][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:13,263][ INFO][PID:2379347] Downloading the builder-live.log file, attempt 3 [2026-01-14 00:01:13,264][ INFO][PID:2379347] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log [2026-01-14 00:01:17,429][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:22,430][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:27,431][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:32,432][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:37,433][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:42,434][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:47,434][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:52,435][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:01:57,436][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:02:02,445][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:02:07,451][ INFO][PID:2379347] Periodic builder liveness probe: dead [2026-01-14 00:02:07,452][ INFO][PID:2379347] calling cancel callback, and waiting [2026-01-14 00:02:08,286][WARNING][PID:2379347] Can't cancel build out: err: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18d' (ED25519) to the list of known hosts. Traceback (most recent call last): File "/usr/bin/copr-rpmbuild-cancel", line 85, in _main() ~~~~~^^ File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main with open(pidfile, "r") as pidfd: ~~~~^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid' [2026-01-14 00:03:15,074][ ERROR][PID:2379347] Re-try request for task on 'ResallocHost, ticket_id=801308, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01214895_20260113_232711, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED [2026-01-14 00:03:15,223][ INFO][PID:2379347] Releasing VM back to pool [2026-01-14 00:03:15,231][ INFO][PID:2379347] Retry #1 (on other host) [2026-01-14 00:03:15,232][ INFO][PID:2379347] Checking that /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2026-01-14 00:03:15,232][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:03:15,233][ INFO][PID:2379347] VM allocation process starts [2026-01-14 00:03:15,289][ INFO][PID:2379347] Trying to allocate VM: ResallocHost, ticket_id=801505, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-14 00:03:18,336][ INFO][PID:2379347] Allocated host ResallocHost, ticket_id=801505, hostname=2620:52:6:1161:dead:beef:cafe:c194, name=vmhost_p09_04_prod_01214993_20260113_233832, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-14 00:03:18,337][ INFO][PID:2379347] Allocating ssh connection to builder [2026-01-14 00:03:18,338][ INFO][PID:2379347] Checking that builder machine is OK [2026-01-14 00:03:19,256][ INFO][PID:2379347] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-14 00:03:19,730][ INFO][PID:2379347] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-14 00:03:19,731][ INFO][PID:2379347] Filling build.info file with builder info [2026-01-14 00:03:19,736][ INFO][PID:2379347] Checking for cancel request [2026-01-14 00:03:19,738][ INFO][PID:2379347] Not re-notifying FE and msg buses for the new host. [2026-01-14 00:03:19,738][ INFO][PID:2379347] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10005412-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-14 00:03:20,683][ INFO][PID:2379347] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c194' (ED25519) to the list of known hosts. [2026-01-14 00:03:20,684][ INFO][PID:2379347] Downloading the builder-live.log file, attempt 1 [2026-01-14 00:03:20,703][ INFO][PID:2379347] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c194 copr-rpmbuild-log [2026-01-14 00:03:25,685][ INFO][PID:2379347] calling cancel callback, and waiting [2026-01-14 00:04:25,745][ ERROR][PID:2379347] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c194 port 22: Connection timed out [2026-01-14 00:05:35,792][ ERROR][PID:2379347] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c194 port 22: Connection timed out [2026-01-14 00:05:49,477][ INFO][PID:2379347] Cancel request succeeded out: success err: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c194' (ED25519) to the list of known hosts. [2026-01-14 00:05:49,478][ INFO][PID:2379347] Downloading results from builder [2026-01-14 00:05:49,478][ INFO][PID:2379347] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c194]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term started [2026-01-14 00:05:49,479][ INFO][PID:2379347] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c194]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/ &> /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/build-10005412.rsync.log [2026-01-14 00:05:50,392][ INFO][PID:2379347] rsyncing finished. [2026-01-14 00:05:50,393][ INFO][PID:2379347] Releasing VM back to pool [2026-01-14 00:05:50,408][ INFO][PID:2379347] Searching for 'success' file in resultdir [2026-01-14 00:05:50,409][ ERROR][PID:2379347] Build failed: Backend process error: No success file => build failure [2026-01-14 00:05:50,410][ INFO][PID:2379347] Finished build: id=10005412 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged chroot=fedora-rawhide-ppc64le [2026-01-14 00:05:50,412][ ERROR][PID:2379347] Backend process error: No success file => build failure [2026-01-14 00:05:50,412][ INFO][PID:2379347] Worker failed build, took 150.67476058006287 [2026-01-14 00:05:50,413][ INFO][PID:2379347] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "ryanabx", "project_name": "cosmic-epoch-tagged", "project_dirname": "cosmic-epoch-tagged", "submitter": "ryanabx", "ended_on": 1768349150.4128978, "started_on": 1768348999.7381372, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10005412-fedora-rawhide-ppc64le", "build_id": 10005412, "package_name": "cosmic-term", "package_version": "1.0.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ryanabx/cosmic-epoch-tagged/cosmic-term", "git_hash": "d1c5c6c97f8d08fa1e559b51cb5505375918f82a", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ryanabx/cosmic-epoch-tagged--ryanabx", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": true, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged", "result_dir": "10005412-cosmic-term", "built_packages": "", "id": 10005412, "mockchain_macros": { "copr_username": "ryanabx", "copr_projectname": "cosmic-epoch-tagged", "vendor": "Fedora Project COPR (ryanabx/cosmic-epoch-tagged)" } } ] } [2026-01-14 00:05:50,461][ INFO][PID:2379347] Sending fedora-messaging bus message in build.end [2026-01-14 00:05:50,489][ INFO][PID:2379347] Compressing /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/builder-live.log by gzip [2026-01-14 00:05:50,490][ INFO][PID:2379347] Running command 'gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/builder-live.log' as PID 2498799 [2026-01-14 00:05:50,506][ INFO][PID:2379347] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/builder-live.log) [2026-01-14 00:05:50,507][ INFO][PID:2379347] Compressing /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/backend.log by gzip [2026-01-14 00:05:50,508][ INFO][PID:2379347] Running command 'gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005412-cosmic-term/backend.log' as PID 2498808