[2025-12-10 22:20:54,916][ INFO][PID:1597377] Marking build as starting [2025-12-10 22:20:54,970][ INFO][PID:1597377] Checking that /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/repodata/repomd.xml exists [2025-12-10 22:20:54,976][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:20:54,977][ INFO][PID:1597377] VM allocation process starts [2025-12-10 22:20:55,000][ INFO][PID:1597377] Trying to allocate VM: ResallocHost, ticket_id=269495, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:21:00,002][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:21:01,022][ INFO][PID:1597377] Allocated host ResallocHost, ticket_id=269495, hostname=2620:52:6:1161:dead:beef:cafe:c10e, name=vmhost_p09_02_prod_00332699_20251210_221309, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:21:01,022][ INFO][PID:1597377] Allocating ssh connection to builder [2025-12-10 22:21:01,023][ INFO][PID:1597377] Checking that builder machine is OK [2025-12-10 22:22:01,090][ ERROR][PID:1597377] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c10e port 22: Connection timed out [2025-12-10 22:22:11,090][ ERROR][PID:1597377] Re-try request for task on 'ResallocHost, ticket_id=269495, hostname=2620:52:6:1161:dead:beef:cafe:c10e, name=vmhost_p09_02_prod_00332699_20251210_221309, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-12-10 22:22:11,091][ INFO][PID:1597377] Releasing VM back to pool [2025-12-10 22:22:11,099][ INFO][PID:1597377] Retry #1 (on other host) [2025-12-10 22:22:11,100][ INFO][PID:1597377] Checking that /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/repodata/repomd.xml exists [2025-12-10 22:22:11,100][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:11,101][ INFO][PID:1597377] VM allocation process starts [2025-12-10 22:22:11,147][ INFO][PID:1597377] Trying to allocate VM: ResallocHost, ticket_id=269519, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:22:14,200][ INFO][PID:1597377] Allocated host ResallocHost, ticket_id=269519, hostname=2620:52:6:1161:dead:beef:cafe:c10a, name=vmhost_p09_02_prod_00332753_20251210_221545, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:22:14,200][ INFO][PID:1597377] Allocating ssh connection to builder [2025-12-10 22:22:14,201][ INFO][PID:1597377] Checking that builder machine is OK [2025-12-10 22:22:15,136][ INFO][PID:1597377] Running remote command: copr-builder-ready epel-10-ppc64le [2025-12-10 22:22:15,599][ INFO][PID:1597377] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-12-10 22:22:15,600][ INFO][PID:1597377] Filling build.info file with builder info [2025-12-10 22:22:15,600][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:15,601][ INFO][PID:1597377] Not re-notifying FE and msg buses for the new host. [2025-12-10 22:22:15,601][ INFO][PID:1597377] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9896656-epel-10-ppc64le --chroot epel-10-ppc64le --detached [2025-12-10 22:22:16,521][ INFO][PID:1597377] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10a' (ED25519) to the list of known hosts. [2025-12-10 22:22:16,521][ INFO][PID:1597377] Downloading the builder-live.log file, attempt 1 [2025-12-10 22:22:16,523][ INFO][PID:1597377] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10a copr-rpmbuild-log [2025-12-10 22:22:21,528][ INFO][PID:1597377] Periodic builder liveness probe: alive [2025-12-10 22:22:21,529][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:26,529][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:31,530][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:36,531][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:41,532][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:46,533][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:51,534][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:22:56,535][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:01,536][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:06,537][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:11,538][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:16,539][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:21,540][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:26,541][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:31,542][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:36,543][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:41,544][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:46,545][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:51,546][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:23:56,547][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:01,548][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:06,549][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:11,550][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:16,551][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:21,552][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:26,553][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:31,554][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:36,555][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:41,556][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:46,557][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:51,557][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:24:56,558][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:01,559][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:06,560][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:11,561][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:16,562][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:21,563][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:26,564][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:31,565][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:36,566][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:41,567][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:46,568][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:51,569][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:25:56,570][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:01,571][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:06,572][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:11,788][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:16,789][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:21,790][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:26,791][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:31,792][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:36,793][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:41,794][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:46,795][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:51,796][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:26:56,797][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:01,798][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:06,799][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:11,800][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:16,801][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:21,801][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:26,802][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:31,803][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:36,804][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:41,806][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:46,807][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:51,808][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:27:56,814][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:01,815][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:06,816][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:11,817][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:16,818][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:21,819][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:26,820][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:31,821][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:36,821][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:41,822][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:46,823][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:51,824][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:28:56,825][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:01,826][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:06,827][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:11,828][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:16,829][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:21,830][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:26,831][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:31,832][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:36,833][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:41,834][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:46,835][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:51,836][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:29:56,837][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:01,837][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:06,838][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:11,839][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:16,841][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:21,842][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:26,842][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:31,843][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:36,844][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:41,845][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:46,846][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:51,847][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:30:56,849][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:01,849][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:06,850][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:11,851][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:16,852][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:21,853][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:26,854][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:31,855][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:36,856][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:41,857][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:46,858][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:51,859][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:31:56,860][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:01,861][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:06,862][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:11,863][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:16,864][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:21,870][ INFO][PID:1597377] Periodic builder liveness probe: alive [2025-12-10 22:32:21,870][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:26,871][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:31,872][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:36,873][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:41,873][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:46,874][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:51,875][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:32:56,876][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:01,877][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:06,878][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:11,879][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:16,880][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:21,881][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:26,882][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:31,883][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:36,884][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:41,885][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:46,886][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:51,887][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:33:56,888][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:01,890][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:06,891][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:11,892][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:16,893][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:21,894][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:26,894][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:31,895][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:36,896][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:41,897][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:46,898][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:51,899][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:34:56,900][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:01,901][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:06,902][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:11,903][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:16,904][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:21,905][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:26,906][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:31,907][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:36,908][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:41,908][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:46,909][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:51,910][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:35:56,912][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:01,912][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:06,913][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:11,914][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:16,915][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:21,917][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:26,918][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:31,919][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:36,920][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:41,921][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:46,922][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:51,924][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:36:56,925][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:01,926][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:06,926][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:11,927][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:16,929][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:21,930][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:26,931][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:31,932][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:36,935][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:42,479][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:49,805][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:37:56,974][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:02,872][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:09,947][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:17,874][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:23,561][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:29,392][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:36,203][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:41,206][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:46,208][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:51,221][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:38:56,228][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:01,239][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:06,242][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:11,245][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:16,247][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:21,252][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:26,254][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:31,256][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:36,258][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:41,267][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:46,270][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:51,274][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:39:56,276][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:01,277][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:06,278][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:11,279][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:16,280][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:21,283][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:26,285][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:31,287][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:36,291][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:41,294][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:46,295][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:51,296][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:40:56,297][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:01,298][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:06,300][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:11,304][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:16,305][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:21,306][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:26,310][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:31,311][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:36,312][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:41,313][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:46,317][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:51,318][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:41:56,319][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:01,320][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:06,322][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:11,323][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:16,324][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:21,326][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:26,349][ INFO][PID:1597377] Periodic builder liveness probe: alive [2025-12-10 22:42:26,350][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:31,351][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:36,352][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:41,353][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:46,358][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:51,363][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:42:56,367][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:01,370][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:06,373][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:11,374][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:16,375][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:21,377][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:26,379][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:31,381][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:36,382][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:41,383][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:46,384][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:51,385][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:43:56,386][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:01,387][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:06,388][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:11,389][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:16,390][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:21,395][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:26,396][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:31,397][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:36,404][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:41,406][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:46,408][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:51,409][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:44:56,410][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:01,415][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:06,418][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:11,420][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:16,423][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:21,424][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:26,426][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:31,427][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:36,429][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:41,431][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:46,433][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:51,447][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:45:56,448][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:01,450][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:06,451][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:11,452][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:16,453][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:21,454][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:26,455][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:31,456][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:36,457][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:41,458][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:46,459][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:51,460][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:46:56,465][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:01,471][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:05,996][ ERROR][PID:1597377] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-10 22:47:06,472][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:11,476][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:15,997][WARNING][PID:1597377] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-10 22:47:16,478][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:21,480][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:25,998][ INFO][PID:1597377] Downloading the builder-live.log file, attempt 2 [2025-12-10 22:47:26,001][ INFO][PID:1597377] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10a copr-rpmbuild-log [2025-12-10 22:47:26,481][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:31,483][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:36,485][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:41,490][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:46,491][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:51,492][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:47:56,494][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:01,497][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:06,498][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:11,499][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:16,501][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:21,509][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:26,513][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:31,516][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:36,517][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:41,518][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:46,519][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:51,520][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:48:56,521][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:01,523][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:06,526][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:11,537][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:16,564][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:21,733][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:26,737][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:27,332][ ERROR][PID:1597377] Re-try request for task on 'ResallocHost, ticket_id=269519, hostname=2620:52:6:1161:dead:beef:cafe:c10a, name=vmhost_p09_02_prod_00332753_20251210_221545, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: copr-rpmbuild-log shouldn't exit != 0 [2025-12-10 22:49:27,332][ INFO][PID:1597377] Releasing VM back to pool [2025-12-10 22:49:27,341][ INFO][PID:1597377] Retry #2 (on other host) [2025-12-10 22:49:27,341][ INFO][PID:1597377] Checking that /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/repodata/repomd.xml exists [2025-12-10 22:49:27,342][ INFO][PID:1597377] Checking for cancel request [2025-12-10 22:49:27,342][ INFO][PID:1597377] VM allocation process starts [2025-12-10 22:49:27,353][ INFO][PID:1597377] Trying to allocate VM: ResallocHost, ticket_id=269766, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:49:30,367][ INFO][PID:1597377] Allocated host ResallocHost, ticket_id=269766, hostname=2620:52:6:1161:dead:beef:cafe:c111, name=vmhost_p09_02_prod_00333074_20251210_224116, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-12-10 22:49:30,367][ INFO][PID:1597377] Allocating ssh connection to builder [2025-12-10 22:49:30,368][ INFO][PID:1597377] Checking that builder machine is OK [2025-12-10 22:50:30,423][ ERROR][PID:1597377] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c111 port 22: Connection timed out [2025-12-10 22:50:40,424][ ERROR][PID:1597377] Re-try request for task on 'ResallocHost, ticket_id=269766, hostname=2620:52:6:1161:dead:beef:cafe:c111, name=vmhost_p09_02_prod_00333074_20251210_224116, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-12-10 22:50:40,424][ INFO][PID:1597377] Releasing VM back to pool [2025-12-10 22:50:40,434][ ERROR][PID:1597377] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c10e', '2620:52:6:1161:dead:beef:cafe:c10a', '2620:52:6:1161:dead:beef:cafe:c111'} [2025-12-10 22:50:40,434][WARNING][PID:1597377] Switching not-finished job state to 'failed' [2025-12-10 22:50:40,435][ INFO][PID:1597377] Worker failed build, took 1704.8333694934845 [2025-12-10 22:50:40,435][ INFO][PID:1597377] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "krouma", "project_name": "golang-1.25", "project_dirname": "golang-1.25", "submitter": "krouma", "ended_on": 1765407040.434669, "started_on": 1765405335.6012995, "submitted_on": null, "status": 0, "chroot": "epel-10-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9896656-epel-10-ppc64le", "build_id": 9896656, "package_name": "golang", "package_version": "1.25.5-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/krouma/golang-1.25/golang", "git_hash": "6a517c91b31d1c22aad552754b6f5f55acdceefe", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "krouma/golang-1.25--krouma", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/krouma/golang-1.25/epel-10-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/krouma/golang-1.25", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/krouma/golang-1.25", "result_dir": "09896656-golang", "built_packages": "", "id": 9896656, "mockchain_macros": { "copr_username": "krouma", "copr_projectname": "golang-1.25", "vendor": "Fedora Project COPR (krouma/golang-1.25)" } } ] } [2025-12-10 22:50:40,475][ INFO][PID:1597377] Sending fedora-messaging bus message in build.end [2025-12-10 22:50:41,001][ INFO][PID:1597377] Compressing /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/09896656-golang/builder-live.log by gzip [2025-12-10 22:50:41,003][ INFO][PID:1597377] Running command 'gzip /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/09896656-golang/builder-live.log' as PID 1723426 [2025-12-10 22:50:41,004][ INFO][PID:1597377] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/09896656-golang/builder-live.log) [2025-12-10 22:50:41,005][ INFO][PID:1597377] Compressing /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/09896656-golang/backend.log by gzip [2025-12-10 22:50:41,006][ INFO][PID:1597377] Running command 'gzip /var/lib/copr/public_html/results/krouma/golang-1.25/epel-10-ppc64le/09896656-golang/backend.log' as PID 1723429