[2025-12-08 20:26:45,974][ INFO][PID:838807] Marking build as starting [2025-12-08 20:26:46,050][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:26:46,052][ INFO][PID:838807] VM allocation process starts [2025-12-08 20:26:46,077][ INFO][PID:838807] Trying to allocate VM: ResallocHost, ticket_id=223772, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:26:49,110][ INFO][PID:838807] Allocated host ResallocHost, ticket_id=223772, hostname=2620:52:6:1161:dead:beef:cafe:c110, name=vmhost_p09_02_prod_00228512_20251208_201605, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:26:49,111][ INFO][PID:838807] Allocating ssh connection to builder [2025-12-08 20:26:49,111][ INFO][PID:838807] Checking that builder machine is OK [2025-12-08 20:26:50,016][ INFO][PID:838807] Running remote command: copr-builder-ready epel-9-ppc64le [2025-12-08 20:29:49,882][ ERROR][PID:838807] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c110' (ED25519) to the list of known hosts. Timeout, server 2620:52:6:1161:dead:beef:cafe:c110 not responding. [2025-12-08 20:29:59,883][ ERROR][PID:838807] Re-try request for task on 'ResallocHost, ticket_id=223772, hostname=2620:52:6:1161:dead:beef:cafe:c110, name=vmhost_p09_02_prod_00228512_20251208_201605, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-12-08 20:29:59,883][ INFO][PID:838807] Releasing VM back to pool [2025-12-08 20:29:59,894][ INFO][PID:838807] Retry #1 (on other host) [2025-12-08 20:29:59,895][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:29:59,895][ INFO][PID:838807] VM allocation process starts [2025-12-08 20:29:59,907][ INFO][PID:838807] Trying to allocate VM: ResallocHost, ticket_id=223855, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:30:02,921][ INFO][PID:838807] Allocated host ResallocHost, ticket_id=223855, hostname=2620:52:6:1161:dead:beef:cafe:c10d, name=vmhost_p09_02_prod_00228653_20251208_202002, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:30:02,921][ INFO][PID:838807] Allocating ssh connection to builder [2025-12-08 20:30:02,922][ INFO][PID:838807] Checking that builder machine is OK [2025-12-08 20:31:02,996][ ERROR][PID:838807] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c10d port 22: Connection timed out [2025-12-08 20:31:12,997][ ERROR][PID:838807] Re-try request for task on 'ResallocHost, ticket_id=223855, hostname=2620:52:6:1161:dead:beef:cafe:c10d, name=vmhost_p09_02_prod_00228653_20251208_202002, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-12-08 20:31:12,998][ INFO][PID:838807] Releasing VM back to pool [2025-12-08 20:31:13,007][ INFO][PID:838807] Retry #2 (on other host) [2025-12-08 20:31:13,008][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:13,008][ INFO][PID:838807] VM allocation process starts [2025-12-08 20:31:13,062][ INFO][PID:838807] Trying to allocate VM: ResallocHost, ticket_id=223876, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:31:16,123][ INFO][PID:838807] Allocated host ResallocHost, ticket_id=223876, hostname=140.211.168.83, name=copr_osuosl_p09_prod_00221982_20251208_165838, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-08 20:31:16,129][ INFO][PID:838807] Allocating ssh connection to builder [2025-12-08 20:31:16,130][ INFO][PID:838807] Checking that builder machine is OK [2025-12-08 20:31:17,478][ INFO][PID:838807] Running remote command: copr-builder-ready epel-9-ppc64le [2025-12-08 20:31:18,465][ INFO][PID:838807] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-12-08 20:31:18,466][ INFO][PID:838807] Filling build.info file with builder info [2025-12-08 20:31:18,467][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:18,467][ INFO][PID:838807] Not re-notifying FE and msg buses for the new host. [2025-12-08 20:31:18,468][ INFO][PID:838807] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9887628-epel-9-ppc64le --chroot epel-9-ppc64le --detached [2025-12-08 20:31:20,061][ INFO][PID:838807] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '140.211.168.83' (ED25519) to the list of known hosts. [2025-12-08 20:31:20,062][ INFO][PID:838807] Downloading the builder-live.log file, attempt 1 [2025-12-08 20:31:20,064][ INFO][PID:838807] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.83 copr-rpmbuild-log [2025-12-08 20:31:25,069][ INFO][PID:838807] Periodic builder liveness probe: alive [2025-12-08 20:31:25,070][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:30,070][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:35,071][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:40,072][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:45,073][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:50,075][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:31:55,076][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:00,077][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:05,078][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:10,078][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:15,079][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:20,081][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:25,082][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:30,083][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:35,084][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:40,085][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:45,086][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:50,087][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:32:55,088][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:00,090][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:05,090][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:10,093][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:15,096][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:20,097][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:25,098][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:30,099][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:35,100][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:40,102][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:45,103][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:50,104][ INFO][PID:838807] Checking for cancel request [2025-12-08 20:33:50,126][ INFO][PID:838807] Downloading results from builder [2025-12-08 20:33:50,127][ INFO][PID:838807] rsyncing of mockbuilder@140.211.168.83:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage started [2025-12-08 20:33:50,128][ INFO][PID:838807] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.83:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/ &> /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/build-09887628.rsync.log [2025-12-08 20:33:51,957][ INFO][PID:838807] rsyncing finished. [2025-12-08 20:33:51,957][ INFO][PID:838807] Releasing VM back to pool [2025-12-08 20:33:51,967][ INFO][PID:838807] Searching for 'success' file in resultdir [2025-12-08 20:33:51,967][ ERROR][PID:838807] Build failed: Backend process error: No success file => build failure [2025-12-08 20:33:51,968][ INFO][PID:838807] Finished build: id=9887628 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/matthewdva/certbot chroot=epel-9-ppc64le [2025-12-08 20:33:51,970][ ERROR][PID:838807] Backend process error: No success file => build failure [2025-12-08 20:33:51,970][ INFO][PID:838807] Worker failed build, took 153.50266218185425 [2025-12-08 20:33:51,971][ INFO][PID:838807] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "matthewdva", "project_name": "certbot", "project_dirname": "certbot", "submitter": "https://src.fedoraproject.org/user/cstratak", "ended_on": 1765226031.9704762, "started_on": 1765225878.467814, "submitted_on": null, "status": 0, "chroot": "epel-9-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9887628-epel-9-ppc64le", "build_id": 9887628, "package_name": "python-SecretStorage", "package_version": "3.5.0-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/matthewdva/certbot/python-SecretStorage", "git_hash": "10fa4dc5924a35b2f7f171cf2b58902d60ed78b0", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "matthewdva/certbot--https://src.fedoraproject.org/user/cstratak", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/matthewdva/certbot/epel-9-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://dl.fedoraproject.org/pub/epel/testing/$releasever/Everything/$basearch", "id": "https_dl_fedoraproject_org_pub_epel_testing_releasever_Everything_basearch", "name": "Additional repo https_dl_fedoraproject_org_pub_epel_testing_releasever_Everything_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/matthewdva/certbot", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/matthewdva/certbot", "result_dir": "09887628-python-SecretStorage", "built_packages": "", "id": 9887628, "mockchain_macros": { "copr_username": "matthewdva", "copr_projectname": "certbot", "vendor": "Fedora Project COPR (matthewdva/certbot)" } } ] } [2025-12-08 20:33:52,004][ INFO][PID:838807] Sending fedora-messaging bus message in build.end [2025-12-08 20:33:52,555][ INFO][PID:838807] Compressing /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/builder-live.log by gzip [2025-12-08 20:33:52,557][ INFO][PID:838807] Running command 'gzip /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/builder-live.log' as PID 892246 [2025-12-08 20:33:52,561][ INFO][PID:838807] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/builder-live.log) [2025-12-08 20:33:52,562][ INFO][PID:838807] Compressing /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/backend.log by gzip [2025-12-08 20:33:52,563][ INFO][PID:838807] Running command 'gzip /var/lib/copr/public_html/results/matthewdva/certbot/epel-9-ppc64le/09887628-python-SecretStorage/backend.log' as PID 892247