[2025-12-11 12:38:16,292][  INFO][PID:2204451] Marking build as starting
[2025-12-11 12:38:16,339][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:38:16,597][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:38:16,824][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:16,826][  INFO][PID:2204451] VM allocation process starts
[2025-12-11 12:38:16,843][  INFO][PID:2204451] Trying to allocate VM: ResallocHost, ticket_id=282974, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:38:21,844][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:22,869][  INFO][PID:2204451] Allocated host ResallocHost, ticket_id=282974, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_00345429_20251211_123217, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:38:22,870][  INFO][PID:2204451] Allocating ssh connection to builder
[2025-12-11 12:38:22,870][  INFO][PID:2204451] Checking that builder machine is OK
[2025-12-11 12:38:23,912][  INFO][PID:2204451] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-11 12:38:24,462][  INFO][PID:2204451] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-11 12:38:24,463][  INFO][PID:2204451] Filling build.info file with builder info
[2025-12-11 12:38:24,463][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:24,464][  INFO][PID:2204451] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@copr",
            "project_name": "copr-pull-requests",
            "project_dirname": "copr-pull-requests:pr:4039",
            "submitter": null,
            "ended_on": null,
            "started_on": 1765456704.4646795,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9899159-fedora-rawhide-ppc64le",
            "build_id": 9899159,
            "package_name": "copr-cli",
            "package_version": "2.4.post1-1.git.0.3ec8522",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4039/copr-cli",
            "git_hash": "358cde681aac596367cefa3996efb7f4f5713536",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@copr/copr-pull-requests--70afeeeb-840e-471a-aed3-4d94466d4421",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/",
                    "id": "copr_coprdir",
                    "name": "Coprdir repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/",
                    "id": "copr_copr_copr_dev",
                    "name": "Additional repo copr_copr_copr_dev"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039",
            "result_dir": "09899159-copr-cli",
            "built_packages": "",
            "id": 9899159,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "copr-pull-requests",
                "vendor": "Fedora Project COPR (@copr/copr-pull-requests)"
            }
        }
    ]
}
[2025-12-11 12:38:24,503][  INFO][PID:2204451] Sending fedora-messaging bus message in build.start
[2025-12-11 12:38:25,123][  INFO][PID:2204451] Sending fedora-messaging bus message in chroot.start
[2025-12-11 12:38:25,159][  INFO][PID:2204451] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9899159-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-11 12:38:27,967][  INFO][PID:2204451] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14e' (ED25519) to the list of known hosts.


[2025-12-11 12:38:27,968][  INFO][PID:2204451] Downloading the builder-live.log file, attempt 1
[2025-12-11 12:38:27,970][  INFO][PID:2204451] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2025-12-11 12:38:32,982][  INFO][PID:2204451] Periodic builder liveness probe: alive
[2025-12-11 12:38:32,983][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:37,985][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:42,986][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:47,987][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:52,989][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:38:57,989][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:02,991][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:07,992][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:12,993][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:17,995][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:22,997][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:27,999][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:33,000][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:38,001][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:43,002][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:48,003][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:53,004][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:39:58,005][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:03,006][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:08,007][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:13,008][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:18,009][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:23,010][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:28,012][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:33,013][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:38,014][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:43,015][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:48,016][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:53,017][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:40:58,018][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:03,019][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:08,020][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:12,370][ ERROR][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-11 12:41:13,021][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:18,022][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:22,371][WARNING][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-11 12:41:23,023][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:28,024][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:32,372][  INFO][PID:2204451] Downloading the builder-live.log file, attempt 2
[2025-12-11 12:41:32,373][  INFO][PID:2204451] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2025-12-11 12:41:33,025][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:38,026][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:43,027][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:48,028][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:53,029][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:41:58,030][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:03,031][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:08,032][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:13,033][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:18,034][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:23,035][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:28,036][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:32,441][ ERROR][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-11 12:42:33,037][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:38,038][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:42,442][WARNING][PID:2204451] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-11 12:42:43,039][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:48,040][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:52,443][  INFO][PID:2204451] Downloading the builder-live.log file, attempt 3
[2025-12-11 12:42:52,444][  INFO][PID:2204451] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2025-12-11 12:42:53,041][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:42:58,042][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:03,043][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:08,044][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:13,045][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:18,046][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:23,047][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:28,048][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:33,049][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:38,050][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:43,051][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:48,056][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:52,513][ ERROR][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-11 12:43:53,057][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:43:58,058][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:02,520][WARNING][PID:2204451] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-11 12:44:03,059][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:08,060][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:12,520][  INFO][PID:2204451] Downloading the builder-live.log file, attempt 4
[2025-12-11 12:44:12,522][  INFO][PID:2204451] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2025-12-11 12:44:13,061][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:18,062][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:23,063][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:28,064][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:33,066][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:38,066][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:43,067][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:48,068][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:53,069][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:44:58,070][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:03,071][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:08,072][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:12,590][ ERROR][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-11 12:45:13,073][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:18,074][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:22,591][ ERROR][PID:2204451] Giving up for unstable SSH, failures: 1765456882.3717356, 1765456962.442626, 1765457042.5200107, 1765457122.5917742
[2025-12-11 12:45:22,592][ ERROR][PID:2204451] Re-try request for task on 'ResallocHost, ticket_id=282974, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_00345429_20251211_123217, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-12-11 12:45:22,593][  INFO][PID:2204451] Releasing VM back to pool
[2025-12-11 12:45:22,604][  INFO][PID:2204451] Retry #1 (on other host)
[2025-12-11 12:45:22,605][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:45:22,843][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:45:23,047][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:45:23,048][  INFO][PID:2204451] VM allocation process starts
[2025-12-11 12:45:23,059][  INFO][PID:2204451] Trying to allocate VM: ResallocHost, ticket_id=283022, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:45:26,072][  INFO][PID:2204451] Allocated host ResallocHost, ticket_id=283022, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00345563_20251211_123617, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:45:26,073][  INFO][PID:2204451] Allocating ssh connection to builder
[2025-12-11 12:45:26,074][  INFO][PID:2204451] Checking that builder machine is OK
[2025-12-11 12:46:26,144][ ERROR][PID:2204451] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18c port 22: Connection timed out

[2025-12-11 12:46:36,144][ ERROR][PID:2204451] Re-try request for task on 'ResallocHost, ticket_id=283022, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00345563_20251211_123617, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-11 12:46:36,145][  INFO][PID:2204451] Releasing VM back to pool
[2025-12-11 12:46:36,157][  INFO][PID:2204451] Retry #2 (on other host)
[2025-12-11 12:46:36,157][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:46:36,345][  INFO][PID:2204451] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le//repodata/repomd.xml exists
[2025-12-11 12:46:36,527][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:46:36,528][  INFO][PID:2204451] VM allocation process starts
[2025-12-11 12:46:36,537][  INFO][PID:2204451] Trying to allocate VM: ResallocHost, ticket_id=283034, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:46:39,551][  INFO][PID:2204451] Allocated host ResallocHost, ticket_id=283034, hostname=2620:52:6:1161:dead:beef:cafe:c191, name=vmhost_p09_04_prod_00345624_20251211_123920, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-11 12:46:39,552][  INFO][PID:2204451] Allocating ssh connection to builder
[2025-12-11 12:46:39,552][  INFO][PID:2204451] Checking that builder machine is OK
[2025-12-11 12:46:40,557][  INFO][PID:2204451] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-11 12:46:41,046][  INFO][PID:2204451] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-11 12:46:41,047][  INFO][PID:2204451] Filling build.info file with builder info
[2025-12-11 12:46:41,050][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:46:41,051][  INFO][PID:2204451] Not re-notifying FE and msg buses for the new host.
[2025-12-11 12:46:41,052][  INFO][PID:2204451] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9899159-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-11 12:46:41,975][  INFO][PID:2204451] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c191' (ED25519) to the list of known hosts.


[2025-12-11 12:46:41,975][  INFO][PID:2204451] Downloading the builder-live.log file, attempt 1
[2025-12-11 12:46:41,976][  INFO][PID:2204451] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c191 copr-rpmbuild-log
[2025-12-11 12:46:46,976][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:46:51,977][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:46:56,978][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:01,979][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:06,980][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:11,981][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:16,982][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:21,983][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:26,984][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:31,985][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:36,986][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:41,986][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:46,987][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:51,989][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:47:56,990][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:01,991][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:06,992][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:11,993][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:16,994][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:21,994][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:26,996][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:31,997][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:37,002][  INFO][PID:2204451] Periodic builder liveness probe: alive
[2025-12-11 12:48:37,003][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:42,004][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:47,005][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:52,006][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:48:57,007][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:02,008][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:07,009][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:12,010][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:17,011][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:22,012][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:27,013][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:32,014][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:37,015][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:42,016][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:47,017][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:52,018][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:49:57,019][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:02,020][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:07,021][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:12,022][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:17,023][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:22,024][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:27,025][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:32,026][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:37,028][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:42,029][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:47,030][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:52,030][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:50:57,031][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:02,032][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:07,033][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:12,034][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:17,036][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:22,037][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:27,038][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:32,039][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:37,040][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:42,040][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:47,042][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:52,043][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:51:57,043][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:02,044][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:07,045][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:12,046][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:17,048][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:22,049][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:27,050][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:32,052][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:37,053][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:42,054][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:47,055][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:52,056][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:52:57,058][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:02,059][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:07,060][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:12,061][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:17,062][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:22,063][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:27,064][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:32,065][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:37,066][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:42,067][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:47,068][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:52,069][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:53:57,071][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:02,072][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:07,072][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:12,074][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:17,075][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:22,076][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:27,077][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:32,079][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:37,080][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:42,081][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:47,083][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:52,084][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:54:57,085][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:02,086][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:07,088][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:12,089][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:17,090][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:22,091][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:27,092][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:32,093][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:37,094][  INFO][PID:2204451] Checking for cancel request
[2025-12-11 12:55:39,099][  INFO][PID:2204451] Downloading results from builder
[2025-12-11 12:55:39,100][  INFO][PID:2204451] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c191]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli started
[2025-12-11 12:55:39,101][  INFO][PID:2204451] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c191]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/ &> /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/build-09899159.rsync.log
[2025-12-11 12:55:39,665][  INFO][PID:2204451] rsyncing finished.
[2025-12-11 12:55:39,666][  INFO][PID:2204451] Releasing VM back to pool
[2025-12-11 12:55:39,682][  INFO][PID:2204451] Searching for 'success' file in resultdir
[2025-12-11 12:55:39,683][  INFO][PID:2204451] Going to sign pkgs from source: 9899159-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le
[2025-12-11 12:55:39,683][  INFO][PID:2204451] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1)
[2025-12-11 12:55:39,948][  INFO][PID:2204451] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.noarch.rpm' (attempt #1)
[2025-12-11 12:55:40,659][  INFO][PID:2204451] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.noarch.rpm
[2025-12-11 12:55:40,660][  INFO][PID:2204451] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.src.rpm' (attempt #1)
[2025-12-11 12:55:41,368][  INFO][PID:2204451] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.src.rpm
[2025-12-11 12:55:41,369][  INFO][PID:2204451] Sign done
[2025-12-11 12:55:41,370][  INFO][PID:2204451] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.noarch.rpm
[2025-12-11 12:55:41,983][  INFO][PID:2204451] [1/2] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.noarch.rpm
[2025-12-11 12:55:41,983][  INFO][PID:2204451] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.src.rpm
[2025-12-11 12:55:42,482][  INFO][PID:2204451] [2/2] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.src.rpm
[2025-12-11 12:55:42,486][  INFO][PID:2204451] Pulp: get_repository: /api/v3/repositories/rpm/rpm/?name=%40copr%2Fcopr-pull-requests%3Apr%3A4039%2Ffedora-rawhide-ppc64le&offset=0&limit=1
[2025-12-11 12:55:42,758][  INFO][PID:2204451] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b0d7b-4b89-76c1-8e10-9169bc16ddaa/
[2025-12-11 12:55:42,759][  INFO][PID:2204451] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b0d7b-4b89-76c1-8e10-9169bc16ddaa/
[2025-12-11 12:55:48,071][  INFO][PID:2204451] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b0d7b-4b89-76c1-8e10-9169bc16ddaa/
[2025-12-11 12:55:48,072][  INFO][PID:2204451] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b0d7b-4b89-76c1-8e10-9169bc16ddaa/
[2025-12-11 12:55:48,203][  INFO][PID:2204451] Successfully modified Pulp repository content /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019aeda5-cf61-7cd2-a209-7d8e11d51fd2/
[2025-12-11 12:55:48,203][  INFO][PID:2204451] Pulp: publishing /api/v3/publications/rpm/rpm/ /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019aeda5-cf61-7cd2-a209-7d8e11d51fd2/
[2025-12-11 12:55:48,363][  INFO][PID:2204451] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b0d7b-6166-7410-bf62-3dbd92b41cfb/
[2025-12-11 12:55:48,363][  INFO][PID:2204451] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b0d7b-6166-7410-bf62-3dbd92b41cfb/
[2025-12-11 12:55:53,557][  INFO][PID:2204451] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b0d7b-6166-7410-bf62-3dbd92b41cfb/
[2025-12-11 12:55:53,558][  INFO][PID:2204451] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b0d7b-6166-7410-bf62-3dbd92b41cfb/
[2025-12-11 12:55:53,795][  INFO][PID:2204451] Successfully created Pulp publication of repository /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019aeda5-cf61-7cd2-a209-7d8e11d51fd2/
[2025-12-11 12:55:53,798][  INFO][PID:2204451] Pulp: add_content: /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019aeda5-cf61-7cd2-a209-7d8e11d51fd2/ (['/api/pulp/public-copr/api/v3/content/rpm/packages/019b0d7b-4863-7db7-a180-89045252c9d4/', '/api/pulp/public-copr/api/v3/content/rpm/packages/019b0d7b-4a42-79b1-bd64-88c083300e88/'])
[2025-12-11 12:55:53,799][  INFO][PID:2204451] Getting build details
[2025-12-11 12:55:53,799][  INFO][PID:2204451] Listing built binary packages in /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli
[2025-12-11 12:55:53,800][  INFO][PID:2204451] Built packages:
['copr-cli 2.4.post1']
[2025-12-11 12:55:53,800][  INFO][PID:2204451] build details: {'built_packages': 'copr-cli 2.4.post1'}
[2025-12-11 12:55:53,800][  INFO][PID:2204451] Retrieving pubkey
[2025-12-11 12:55:53,801][  INFO][PID:2204451] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1)
[2025-12-11 12:55:54,070][  INFO][PID:2204451] Added pubkey for user @copr project copr-pull-requests into: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/pubkey.gpg
[2025-12-11 12:55:54,071][  INFO][PID:2204451] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.noarch.rpm, it is stored in Pulp
[2025-12-11 12:55:54,072][  INFO][PID:2204451] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/copr-cli-2.4.post1-1.git.0.3ec8522.fc44.src.rpm, it is stored in Pulp
[2025-12-11 12:55:54,072][  INFO][PID:2204451] Finished build: id=9899159 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039 chroot=fedora-rawhide-ppc64le 
[2025-12-11 12:55:54,074][  INFO][PID:2204451] Worker succeeded build, took 553.022497177124
[2025-12-11 12:55:54,075][  INFO][PID:2204451] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@copr",
            "project_name": "copr-pull-requests",
            "project_dirname": "copr-pull-requests:pr:4039",
            "submitter": null,
            "ended_on": 1765457754.0743804,
            "started_on": 1765457201.0518832,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9899159-fedora-rawhide-ppc64le",
            "build_id": 9899159,
            "package_name": "copr-cli",
            "package_version": "2.4.post1-1.git.0.3ec8522",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4039/copr-cli",
            "git_hash": "358cde681aac596367cefa3996efb7f4f5713536",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@copr/copr-pull-requests--70afeeeb-840e-471a-aed3-4d94466d4421",
            "results": {
                "packages": [
                    {
                        "name": "copr-cli",
                        "epoch": null,
                        "version": "2.4.post1",
                        "release": "1.git.0.3ec8522.fc44",
                        "arch": "noarch"
                    },
                    {
                        "name": "copr-cli",
                        "epoch": null,
                        "version": "2.4.post1",
                        "release": "1.git.0.3ec8522.fc44",
                        "arch": "src"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/",
                    "id": "copr_coprdir",
                    "name": "Coprdir repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/",
                    "id": "copr_copr_copr_dev",
                    "name": "Additional repo copr_copr_copr_dev"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4039",
            "result_dir": "09899159-copr-cli",
            "built_packages": "copr-cli 2.4.post1",
            "id": 9899159,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "copr-pull-requests",
                "vendor": "Fedora Project COPR (@copr/copr-pull-requests)"
            }
        }
    ]
}
[2025-12-11 12:55:54,119][  INFO][PID:2204451] Sending fedora-messaging bus message in build.end
[2025-12-11 12:55:54,149][  INFO][PID:2204451] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/builder-live.log by gzip
[2025-12-11 12:55:54,150][  INFO][PID:2204451] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/builder-live.log' as PID 2287128
[2025-12-11 12:55:54,157][  INFO][PID:2204451] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/builder-live.log)
[2025-12-11 12:55:54,158][  INFO][PID:2204451] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/backend.log by gzip
[2025-12-11 12:55:54,159][  INFO][PID:2204451] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4039/fedora-rawhide-ppc64le/09899159-copr-cli/backend.log' as PID 2287129