[2026-01-08 02:27:14,709][ INFO][PID:3285068] Marking build as starting [2026-01-08 02:27:14,927][ INFO][PID:3285068] Checking that /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/repodata/repomd.xml exists [2026-01-08 02:27:14,930][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:14,931][ INFO][PID:3285068] VM allocation process starts [2026-01-08 02:27:14,949][ INFO][PID:3285068] Trying to allocate VM: ResallocHost, ticket_id=703083, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-08 02:27:19,951][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:24,952][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:29,954][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:34,958][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:39,959][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:44,960][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:49,962][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:54,964][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:27:59,966][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:04,967][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:09,968][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:14,969][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:19,970][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:24,971][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:29,972][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:34,973][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:39,974][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:44,975][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:49,976][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:54,977][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:28:59,979][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:04,981][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:09,983][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:14,984][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:19,985][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:24,986][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:29,987][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:34,988][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:39,989][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:44,991][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:49,993][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:54,994][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:29:59,995][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:04,996][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:09,997][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:14,998][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:19,999][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:25,001][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:30,002][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:30:33,084][ INFO][PID:3285068] Allocated host ResallocHost, ticket_id=703083, hostname=2620:52:6:1161:dead:beef:cafe:c14b, name=vmhost_p09_03_prod_01007797_20260108_022324, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-08 02:30:33,085][ INFO][PID:3285068] Allocating ssh connection to builder [2026-01-08 02:30:33,085][ INFO][PID:3285068] Checking that builder machine is OK [2026-01-08 02:30:33,942][ INFO][PID:3285068] Running remote command: copr-builder-ready fedora-42-ppc64le [2026-01-08 02:31:34,011][ ERROR][PID:3285068] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14b port 22: Connection timed out [2026-01-08 02:31:44,012][ ERROR][PID:3285068] Re-try request for task on 'ResallocHost, ticket_id=703083, hostname=2620:52:6:1161:dead:beef:cafe:c14b, name=vmhost_p09_03_prod_01007797_20260108_022324, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2026-01-08 02:31:44,013][ INFO][PID:3285068] Releasing VM back to pool [2026-01-08 02:31:44,025][ INFO][PID:3285068] Retry #1 (on other host) [2026-01-08 02:31:44,025][ INFO][PID:3285068] Checking that /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/repodata/repomd.xml exists [2026-01-08 02:31:44,026][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:31:44,026][ INFO][PID:3285068] VM allocation process starts [2026-01-08 02:31:44,037][ INFO][PID:3285068] Trying to allocate VM: ResallocHost, ticket_id=703188, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-08 02:31:49,038][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:31:54,040][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:31:59,047][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:04,048][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:09,050][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:14,051][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:19,052][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:22,086][ INFO][PID:3285068] Allocated host ResallocHost, ticket_id=703188, hostname=2620:52:6:1161:dead:beef:cafe:c156, name=vmhost_p09_03_prod_01007468_20260108_020649, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-08 02:32:22,087][ INFO][PID:3285068] Allocating ssh connection to builder [2026-01-08 02:32:22,087][ INFO][PID:3285068] Checking that builder machine is OK [2026-01-08 02:32:22,536][ INFO][PID:3285068] Running remote command: copr-builder-ready fedora-42-ppc64le [2026-01-08 02:32:22,988][ INFO][PID:3285068] Red Hat subscription not needed for fedora-42-ppc64le Builder is ready to be used [2026-01-08 02:32:22,989][ INFO][PID:3285068] Filling build.info file with builder info [2026-01-08 02:32:22,990][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:22,991][ INFO][PID:3285068] Not re-notifying FE and msg buses for the new host. [2026-01-08 02:32:22,992][ INFO][PID:3285068] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9983709-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached [2026-01-08 02:32:23,809][ INFO][PID:3285068] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c156' (ED25519) to the list of known hosts. [2026-01-08 02:32:23,810][ INFO][PID:3285068] Downloading the builder-live.log file, attempt 1 [2026-01-08 02:32:23,812][ INFO][PID:3285068] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c156 copr-rpmbuild-log [2026-01-08 02:32:28,825][ INFO][PID:3285068] Periodic builder liveness probe: alive [2026-01-08 02:32:28,825][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:33,826][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:38,828][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:43,829][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:48,830][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:53,832][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:32:58,833][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:03,833][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:08,834][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:13,836][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:18,837][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:23,838][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:28,840][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:33,841][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:38,843][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:43,847][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:48,850][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:53,851][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:33:58,852][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:03,853][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:08,854][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:13,855][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:18,857][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:23,858][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:28,859][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:33,903][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:38,904][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:43,907][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:48,909][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:53,917][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:34:58,940][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:35:03,944][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:35:08,945][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:35:13,946][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:35:18,947][ INFO][PID:3285068] Checking for cancel request [2026-01-08 02:35:23,338][ INFO][PID:3285068] Downloading results from builder [2026-01-08 02:35:23,339][ INFO][PID:3285068] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c156]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf started [2026-01-08 02:35:23,340][ INFO][PID:3285068] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c156]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/ &> /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/build-09983709.rsync.log [2026-01-08 02:35:24,050][ INFO][PID:3285068] rsyncing finished. [2026-01-08 02:35:24,050][ INFO][PID:3285068] Releasing VM back to pool [2026-01-08 02:35:24,065][ INFO][PID:3285068] Searching for 'success' file in resultdir [2026-01-08 02:35:24,066][ INFO][PID:3285068] Going to sign pkgs from source: 9983709-fedora-42-ppc64le in chroot: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le [2026-01-08 02:35:24,067][ INFO][PID:3285068] Calling '/bin/sign -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1) [2026-01-08 02:35:24,344][ INFO][PID:3285068] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-bootc-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm' (attempt #1) [2026-01-08 02:35:25,006][ INFO][PID:3285068] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-bootc-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm [2026-01-08 02:35:25,006][ INFO][PID:3285068] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.src.rpm' (attempt #1) [2026-01-08 02:35:25,924][ INFO][PID:3285068] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.src.rpm [2026-01-08 02:35:25,925][ INFO][PID:3285068] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/python3-dnf-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm' (attempt #1) [2026-01-08 02:35:26,819][ INFO][PID:3285068] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/python3-dnf-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm [2026-01-08 02:35:26,820][ INFO][PID:3285068] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-data-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm' (attempt #1) [2026-01-08 02:35:27,655][ INFO][PID:3285068] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/dnf-data-4.25.0-1.20260108020930993516.master.9.gba14891d.fc42.noarch.rpm [2026-01-08 02:35:27,656][ INFO][PID:3285068] Sign done [2026-01-08 02:35:27,656][ INFO][PID:3285068] Incremental createrepo run, adding 09983709-dnf into https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le, (auto-create-repo=True) [2026-01-08 02:35:27,657][ INFO][PID:3285068] Running command 'copr-repo --batched /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le --add 09983709-dnf --no-appstream-metadata' as PID 3340857 [2026-01-08 02:35:29,096][ INFO][PID:3285068] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le --add 09983709-dnf --no-appstream-metadata) [2026-01-08 02:35:29,097][ INFO][PID:3285068] Getting build details [2026-01-08 02:35:29,097][ INFO][PID:3285068] Listing built binary packages in /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf [2026-01-08 02:35:29,098][ INFO][PID:3285068] Built packages: ['python3-dnf 4.25.0', 'dnf-data 4.25.0', 'dnf-bootc 4.25.0'] [2026-01-08 02:35:29,098][ INFO][PID:3285068] build details: {'built_packages': 'python3-dnf 4.25.0\ndnf-data 4.25.0\ndnf-bootc 4.25.0'} [2026-01-08 02:35:29,099][ INFO][PID:3285068] Retrieving pubkey [2026-01-08 02:35:29,099][ INFO][PID:3285068] Calling '/bin/sign -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1) [2026-01-08 02:35:29,378][ INFO][PID:3285068] Added pubkey for user amatej project dnf-nightly-sanitizers into: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/pubkey.gpg [2026-01-08 02:35:29,379][ INFO][PID:3285068] Finished build: id=9983709 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers chroot=fedora-42-ppc64le [2026-01-08 02:35:29,381][ INFO][PID:3285068] Worker succeeded build, took 186.3901195526123 [2026-01-08 02:35:29,381][ INFO][PID:3285068] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "amatej", "project_name": "dnf-nightly-sanitizers", "project_dirname": "dnf-nightly-sanitizers", "submitter": "amatej", "ended_on": 1767839729.3812885, "started_on": 1767839542.991169, "submitted_on": null, "status": 1, "chroot": "fedora-42-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9983709-fedora-42-ppc64le", "build_id": 9983709, "package_name": "dnf", "package_version": "4.25.0-1.20260108020930993516.master.9.gba14891d", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/amatej/dnf-nightly-sanitizers/dnf", "git_hash": "f35b634a6c3c4ffcc47db30a30a8318ade12622e", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "amatej/dnf-nightly-sanitizers--amatej", "results": { "packages": [ { "name": "dnf", "epoch": null, "version": "4.25.0", "release": "1.20260108020930993516.master.9.gba14891d.fc42", "arch": "src" }, { "name": "python3-dnf", "epoch": null, "version": "4.25.0", "release": "1.20260108020930993516.master.9.gba14891d.fc42", "arch": "noarch" }, { "name": "dnf-data", "epoch": null, "version": "4.25.0", "release": "1.20260108020930993516.master.9.gba14891d.fc42", "arch": "noarch" }, { "name": "dnf-bootc", "epoch": null, "version": "4.25.0", "release": "1.20260108020930993516.master.9.gba14891d.fc42", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers", "result_dir": "09983709-dnf", "built_packages": "python3-dnf 4.25.0\ndnf-data 4.25.0\ndnf-bootc 4.25.0", "id": 9983709, "mockchain_macros": { "copr_username": "amatej", "copr_projectname": "dnf-nightly-sanitizers", "vendor": "Fedora Project COPR (amatej/dnf-nightly-sanitizers)" } } ] } [2026-01-08 02:35:29,450][ INFO][PID:3285068] Sending fedora-messaging bus message in build.end [2026-01-08 02:35:29,967][ INFO][PID:3285068] Compressing /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/builder-live.log by gzip [2026-01-08 02:35:29,969][ INFO][PID:3285068] Running command 'gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/builder-live.log' as PID 3341092 [2026-01-08 02:35:30,017][ INFO][PID:3285068] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/builder-live.log) [2026-01-08 02:35:30,018][ INFO][PID:3285068] Compressing /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/backend.log by gzip [2026-01-08 02:35:30,019][ INFO][PID:3285068] Running command 'gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-42-ppc64le/09983709-dnf/backend.log' as PID 3341096