[2026-01-14 07:37:17,275][  INFO][PID:471222] Marking build as starting
[2026-01-14 07:37:17,460][  INFO][PID:471222] Checking that /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 07:37:17,497][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:37:17,498][  INFO][PID:471222] VM allocation process starts
[2026-01-14 07:37:17,577][  INFO][PID:471222] Trying to allocate VM: ResallocHost, ticket_id=805670, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:37:20,612][  INFO][PID:471222] Allocated host ResallocHost, ticket_id=805670, hostname=2620:52:6:1161:dead:beef:cafe:c192, name=vmhost_p09_04_prod_01219048_20260114_072217, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:37:20,612][  INFO][PID:471222] Allocating ssh connection to builder
[2026-01-14 07:37:20,613][  INFO][PID:471222] Checking that builder machine is OK
[2026-01-14 07:37:21,598][  INFO][PID:471222] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-14 07:38:21,667][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c192 port 22: Connection timed out

[2026-01-14 07:38:31,668][ ERROR][PID:471222] Re-try request for task on 'ResallocHost, ticket_id=805670, hostname=2620:52:6:1161:dead:beef:cafe:c192, name=vmhost_p09_04_prod_01219048_20260114_072217, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-14 07:38:31,668][  INFO][PID:471222] Releasing VM back to pool
[2026-01-14 07:38:31,680][  INFO][PID:471222] Retry #1 (on other host)
[2026-01-14 07:38:31,680][  INFO][PID:471222] Checking that /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 07:38:31,681][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:38:31,681][  INFO][PID:471222] VM allocation process starts
[2026-01-14 07:38:31,692][  INFO][PID:471222] Trying to allocate VM: ResallocHost, ticket_id=805691, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:38:34,702][  INFO][PID:471222] Allocated host ResallocHost, ticket_id=805691, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_01219156_20260114_073335, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:38:34,703][  INFO][PID:471222] Allocating ssh connection to builder
[2026-01-14 07:38:34,703][  INFO][PID:471222] Checking that builder machine is OK
[2026-01-14 07:39:34,771][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18a port 22: Connection timed out

[2026-01-14 07:39:44,772][ ERROR][PID:471222] Re-try request for task on 'ResallocHost, ticket_id=805691, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_01219156_20260114_073335, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-14 07:39:44,772][  INFO][PID:471222] Releasing VM back to pool
[2026-01-14 07:39:44,785][  INFO][PID:471222] Retry #2 (on other host)
[2026-01-14 07:39:44,786][  INFO][PID:471222] Checking that /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 07:39:44,786][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:39:44,787][  INFO][PID:471222] VM allocation process starts
[2026-01-14 07:39:44,798][  INFO][PID:471222] Trying to allocate VM: ResallocHost, ticket_id=805705, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:39:47,810][  INFO][PID:471222] Allocated host ResallocHost, ticket_id=805705, hostname=2620:52:6:1161:dead:beef:cafe:c151, name=vmhost_p09_03_prod_01219160_20260114_073427, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 07:39:47,810][  INFO][PID:471222] Allocating ssh connection to builder
[2026-01-14 07:39:47,810][  INFO][PID:471222] Checking that builder machine is OK
[2026-01-14 07:39:48,743][  INFO][PID:471222] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-14 07:39:49,217][  INFO][PID:471222] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2026-01-14 07:39:49,218][  INFO][PID:471222] Filling build.info file with builder info
[2026-01-14 07:39:49,219][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:39:49,220][  INFO][PID:471222] Not re-notifying FE and msg buses for the new host.
[2026-01-14 07:39:49,220][  INFO][PID:471222] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10006515-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2026-01-14 07:39:50,278][  INFO][PID:471222] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c151' (ED25519) to the list of known hosts.


[2026-01-14 07:39:50,279][  INFO][PID:471222] Downloading the builder-live.log file, attempt 1
[2026-01-14 07:39:50,280][  INFO][PID:471222] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c151 copr-rpmbuild-log
[2026-01-14 07:39:55,284][  INFO][PID:471222] Periodic builder liveness probe: alive
[2026-01-14 07:39:55,285][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:00,286][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:05,287][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:10,288][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:15,289][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:20,290][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:25,291][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:30,292][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:35,292][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:40,293][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:45,294][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:50,295][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:40:55,296][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:00,297][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:05,299][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:10,300][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:15,301][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:20,302][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:25,302][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:30,303][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:35,304][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:40,305][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:45,306][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:50,307][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:41:55,308][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:00,309][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:05,310][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:10,311][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:15,312][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:20,313][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:25,314][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:30,317][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:35,318][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:40,319][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:45,320][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:50,321][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:42:55,323][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:00,324][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:05,325][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:10,326][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:15,327][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:20,328][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:25,329][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:30,330][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:35,331][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:40,332][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:45,332][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:50,334][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:43:55,335][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:00,336][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:05,337][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:10,338][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:15,338][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:20,339][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:25,340][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:30,341][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:35,342][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:40,344][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:45,345][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:50,346][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:44:55,347][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:00,348][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:05,349][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:10,350][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:15,351][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:20,352][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:25,353][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:30,354][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:35,355][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:40,356][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:45,357][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:50,358][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:45:55,359][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:00,360][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:05,361][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:10,362][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:15,363][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:20,417][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:25,418][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:30,419][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:35,420][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:40,421][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:45,422][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:50,423][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:46:55,440][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:00,495][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:05,496][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:10,497][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:15,498][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:20,499][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:25,500][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:30,501][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:35,502][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:40,503][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:45,535][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:50,535][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:47:55,536][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:00,537][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:05,538][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:10,539][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:15,540][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:20,548][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:25,549][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:30,550][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:35,551][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:40,552][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:45,553][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:50,554][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:48:55,555][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:00,556][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:05,557][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:10,558][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:15,558][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:20,559][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:25,560][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:30,561][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:35,563][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:40,564][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:45,565][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:50,566][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:49:55,594][  INFO][PID:471222] Periodic builder liveness probe: alive
[2026-01-14 07:49:55,595][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:00,596][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:05,597][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:10,628][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:15,630][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:20,632][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:25,634][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:30,636][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:35,644][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:40,646][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:45,648][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:50,649][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:50:55,654][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:00,656][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:05,666][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:10,669][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:15,670][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:20,671][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:25,688][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:30,716][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:35,718][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:40,719][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:45,722][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:50,723][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:51:55,724][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:00,725][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:05,727][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:10,728][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:15,814][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:20,815][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:25,816][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:30,817][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:35,818][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:40,819][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:45,829][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:50,831][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:52:55,836][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:00,837][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:05,838][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:10,961][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:15,963][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:20,964][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:25,966][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:30,967][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:36,055][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:41,135][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:46,142][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:51,148][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:53:56,157][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:01,165][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:06,166][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:11,168][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:16,170][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:21,173][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:25,370][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-14 07:54:26,175][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:31,428][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:35,371][WARNING][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-14 07:54:36,433][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:41,438][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:45,372][  INFO][PID:471222] Downloading the builder-live.log file, attempt 2
[2026-01-14 07:54:45,374][  INFO][PID:471222] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c151 copr-rpmbuild-log
[2026-01-14 07:54:46,440][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:51,443][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:54:56,444][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:01,446][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:06,448][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:11,449][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:16,451][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:21,452][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:26,453][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:31,456][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:36,461][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:41,462][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:45,443][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-14 07:55:46,464][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:51,466][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:55:55,458][WARNING][PID:471222] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-14 07:55:56,467][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:01,468][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:05,459][  INFO][PID:471222] Downloading the builder-live.log file, attempt 3
[2026-01-14 07:56:05,461][  INFO][PID:471222] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c151 copr-rpmbuild-log
[2026-01-14 07:56:06,470][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:11,475][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:16,476][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:21,477][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:26,479][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:31,480][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:36,482][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:41,483][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:46,484][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:51,485][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:56:56,486][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:01,487][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:05,523][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-14 07:57:06,505][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:11,507][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:15,531][WARNING][PID:471222] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-14 07:57:16,509][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:21,513][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:25,531][  INFO][PID:471222] Downloading the builder-live.log file, attempt 4
[2026-01-14 07:57:25,533][  INFO][PID:471222] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c151 copr-rpmbuild-log
[2026-01-14 07:57:26,515][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:31,516][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:36,676][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:41,678][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:46,875][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:51,876][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:57:56,877][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:01,878][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:07,065][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:12,067][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:17,409][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:22,410][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:25,598][ ERROR][PID:471222] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-14 07:58:27,411][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:32,412][  INFO][PID:471222] Checking for cancel request
[2026-01-14 07:58:35,609][ ERROR][PID:471222] Giving up for unstable SSH, failures: 1768377275.3712957, 1768377355.4589405, 1768377435.5309937, 1768377515.6089365
[2026-01-14 07:58:35,609][ ERROR][PID:471222] Re-try request for task on 'ResallocHost, ticket_id=805705, hostname=2620:52:6:1161:dead:beef:cafe:c151, name=vmhost_p09_03_prod_01219160_20260114_073427, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-01-14 07:58:35,610][  INFO][PID:471222] Releasing VM back to pool
[2026-01-14 07:58:35,618][ ERROR][PID:471222] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c192', '2620:52:6:1161:dead:beef:cafe:c151', '2620:52:6:1161:dead:beef:cafe:c18a'}
[2026-01-14 07:58:35,619][WARNING][PID:471222] Switching not-finished job state to 'failed'
[2026-01-14 07:58:35,619][  INFO][PID:471222] Worker failed build, took 1126.3987667560577
[2026-01-14 07:58:35,619][  INFO][PID:471222] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "music",
            "project_name": "abseil-cpp",
            "project_dirname": "abseil-cpp",
            "submitter": "music",
            "ended_on": 1768377515.6191015,
            "started_on": 1768376389.2203348,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10006515-fedora-rawhide-ppc64le",
            "build_id": 10006515,
            "package_name": "buildbox",
            "package_version": "1.3.44-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/music/abseil-cpp/buildbox",
            "git_hash": "470d88be99c498c68cc1134f1591b21debef8bcf",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "music/abseil-cpp--music",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/music/abseil-cpp/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/music/abseil-cpp",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/music/abseil-cpp",
            "result_dir": "10006515-buildbox",
            "built_packages": "",
            "id": 10006515,
            "mockchain_macros": {
                "copr_username": "music",
                "copr_projectname": "abseil-cpp",
                "vendor": "Fedora Project COPR (music/abseil-cpp)"
            }
        }
    ]
}
[2026-01-14 07:58:35,659][  INFO][PID:471222] Sending fedora-messaging bus message in build.end
[2026-01-14 07:58:36,185][  INFO][PID:471222] Compressing /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/10006515-buildbox/builder-live.log by gzip
[2026-01-14 07:58:36,187][  INFO][PID:471222] Running command 'gzip /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/10006515-buildbox/builder-live.log' as PID 587255
[2026-01-14 07:58:36,188][  INFO][PID:471222] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/10006515-buildbox/builder-live.log)
[2026-01-14 07:58:36,188][  INFO][PID:471222] Compressing /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/10006515-buildbox/backend.log by gzip
[2026-01-14 07:58:36,189][  INFO][PID:471222] Running command 'gzip /var/lib/copr/public_html/results/music/abseil-cpp/fedora-rawhide-ppc64le/10006515-buildbox/backend.log' as PID 587256