[2025-07-23 02:54:13,984][  INFO][PID:2313867] Marking build as starting
[2025-07-23 02:54:14,063][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:14,075][  INFO][PID:2313867] VM allocation process starts
[2025-07-23 02:54:14,113][  INFO][PID:2313867] Trying to allocate VM: ResallocHost, ticket_id=4048345, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 02:54:17,155][  INFO][PID:2313867] Allocated host ResallocHost, ticket_id=4048345, hostname=140.211.168.8, name=copr_osuosl_p09_prod_05747435_20250723_022143, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 02:54:17,155][  INFO][PID:2313867] Allocating ssh connection to builder
[2025-07-23 02:54:17,157][  INFO][PID:2313867] Checking that builder machine is OK
[2025-07-23 02:54:18,520][  INFO][PID:2313867] Running remote command: copr-builder-ready fedora-eln-ppc64le
[2025-07-23 02:54:19,571][  INFO][PID:2313867] Red Hat subscription not needed for fedora-eln-ppc64le
Builder is ready to be used

[2025-07-23 02:54:19,571][  INFO][PID:2313867] Filling build.info file with builder info
[2025-07-23 02:54:19,572][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:19,573][  INFO][PID:2313867] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "rpmsoftwaremanagement",
            "project_name": "dnf-nightly",
            "project_dirname": "dnf-nightly",
            "submitter": "rpmsoftwaremanagement",
            "ended_on": null,
            "started_on": 1753239259.5732055,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-eln-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9306944-fedora-eln-ppc64le",
            "build_id": 9306944,
            "package_name": "microdnf",
            "package_version": "3.10.1-20250723012110.1.g26c9397",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rpmsoftwaremanagement/dnf-nightly/microdnf",
            "git_hash": "cd40f95006bd643321ae28bdb94eca4be247f6d8",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rpmsoftwaremanagement/dnf-nightly--rpmsoftwaremanagement",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://kojipkgs.fedoraproject.org/repos/eln-build/latest/$basearch",
                    "id": "https_kojipkgs_fedoraproject_org_repos_eln_build_latest_basearch",
                    "name": "Additional repo https_kojipkgs_fedoraproject_org_repos_eln_build_latest_basearch"
                }
            ],
            "background": false,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly",
            "result_dir": "09306944-microdnf",
            "built_packages": "",
            "id": 9306944,
            "mockchain_macros": {
                "copr_username": "rpmsoftwaremanagement",
                "copr_projectname": "dnf-nightly",
                "vendor": "Fedora Project COPR (rpmsoftwaremanagement/dnf-nightly)"
            }
        }
    ]
}
[2025-07-23 02:54:19,617][  INFO][PID:2313867] Sending fedora-messaging bus message in build.start
[2025-07-23 02:54:20,624][  INFO][PID:2313867] Sending fedora-messaging bus message in chroot.start
[2025-07-23 02:54:20,667][  INFO][PID:2313867] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9306944-fedora-eln-ppc64le --chroot fedora-eln-ppc64le --detached
[2025-07-23 02:54:22,038][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 1
[2025-07-23 02:54:22,042][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.8 copr-rpmbuild-log
[2025-07-23 02:54:27,072][  INFO][PID:2313867] Periodic builder liveness probe: alive
[2025-07-23 02:54:27,081][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:32,093][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:37,094][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:42,097][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:47,100][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:52,102][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:54:57,105][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:02,110][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:07,115][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:12,117][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:17,122][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:22,129][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:27,130][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:32,137][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:37,155][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:42,158][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:47,163][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:52,165][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:55:57,183][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:02,187][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:07,197][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:12,198][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:17,204][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:22,206][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:27,211][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:32,221][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:37,226][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:42,237][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:47,242][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:52,258][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:56:57,261][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:02,264][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:07,276][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:12,283][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:17,297][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:22,303][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:27,306][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:32,311][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:37,312][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:42,324][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:47,328][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:52,334][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:57:57,338][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:02,344][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:07,348][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:12,354][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:17,356][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:22,361][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:27,366][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:32,378][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:37,380][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:42,385][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:47,391][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:52,392][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:58:57,395][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:02,401][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:07,402][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:12,406][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:17,411][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:22,412][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:27,414][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:32,422][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:37,436][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:42,440][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:44,143][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 02:59:47,450][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:52,455][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 02:59:54,151][WARNING][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 02:59:57,456][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:02,463][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:04,152][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 2
[2025-07-23 03:00:04,160][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.8 copr-rpmbuild-log
[2025-07-23 03:00:07,465][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:12,473][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:17,479][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:22,485][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:27,491][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:32,500][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:37,505][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:42,507][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:47,509][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:52,515][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:00:57,525][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:02,537][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:04,317][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:01:07,540][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:12,544][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:14,320][WARNING][PID:2313867] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:01:17,552][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:22,553][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:24,323][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 3
[2025-07-23 03:01:24,327][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.8 copr-rpmbuild-log
[2025-07-23 03:01:27,554][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:32,556][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:37,557][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:42,560][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:47,561][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:52,562][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:01:57,565][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:02,566][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:07,568][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:12,575][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:17,577][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:22,581][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:24,465][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:02:27,587][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:32,592][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:34,466][WARNING][PID:2313867] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:02:37,596][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:42,597][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:44,467][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 4
[2025-07-23 03:02:44,469][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.8 copr-rpmbuild-log
[2025-07-23 03:02:47,599][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:52,601][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:02:57,604][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:02,605][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:07,608][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:12,611][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:17,613][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:22,614][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:27,619][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:32,620][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:37,621][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:42,622][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:44,607][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:03:47,624][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:52,626][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:54,610][ ERROR][PID:2313867] Giving up for unstable SSH, failures: 1753239594.1511278, 1753239674.3202386, 1753239754.4663012, 1753239834.6099477
[2025-07-23 03:03:54,613][ ERROR][PID:2313867] Re-try request for task on 'ResallocHost, ticket_id=4048345, hostname=140.211.168.8, name=copr_osuosl_p09_prod_05747435_20250723_022143, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-23 03:03:54,615][  INFO][PID:2313867] Releasing VM back to pool
[2025-07-23 03:03:54,632][  INFO][PID:2313867] Retry #1 (on other host)
[2025-07-23 03:03:54,634][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:03:54,637][  INFO][PID:2313867] VM allocation process starts
[2025-07-23 03:03:54,685][  INFO][PID:2313867] Trying to allocate VM: ResallocHost, ticket_id=4048609, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 03:03:57,759][  INFO][PID:2313867] Allocated host ResallocHost, ticket_id=4048609, hostname=140.211.168.199, name=copr_osuosl_p09_prod_05748103_20250723_024739, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 03:03:57,760][  INFO][PID:2313867] Allocating ssh connection to builder
[2025-07-23 03:03:57,760][  INFO][PID:2313867] Checking that builder machine is OK
[2025-07-23 03:03:59,201][  INFO][PID:2313867] Running remote command: copr-builder-ready fedora-eln-ppc64le
[2025-07-23 03:04:00,263][  INFO][PID:2313867] Red Hat subscription not needed for fedora-eln-ppc64le
Builder is ready to be used

[2025-07-23 03:04:00,263][  INFO][PID:2313867] Filling build.info file with builder info
[2025-07-23 03:04:00,270][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:00,271][  INFO][PID:2313867] Not re-notifying FE and msg buses for the new host.
[2025-07-23 03:04:00,272][  INFO][PID:2313867] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9306944-fedora-eln-ppc64le --chroot fedora-eln-ppc64le --detached
[2025-07-23 03:04:01,608][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 1
[2025-07-23 03:04:01,610][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.199 copr-rpmbuild-log
[2025-07-23 03:04:06,609][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:11,612][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:16,615][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:21,619][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:26,620][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:31,630][  INFO][PID:2313867] Periodic builder liveness probe: alive
[2025-07-23 03:04:31,631][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:36,632][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:41,634][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:46,635][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:51,637][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:04:56,638][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:01,641][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:06,644][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:11,646][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:16,647][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:21,649][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:26,650][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:31,654][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:36,657][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:41,658][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:46,659][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:51,661][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:05:56,662][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:01,663][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:06,665][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:11,671][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:16,673][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:21,677][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:26,680][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:31,682][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:36,683][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:41,685][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:46,686][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:51,689][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:06:56,693][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:01,694][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:06,698][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:11,703][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:16,704][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:21,709][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:26,713][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:31,715][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:36,723][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:41,725][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:46,728][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:51,730][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:07:56,732][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:01,734][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:06,738][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:11,741][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:16,744][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:21,747][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:26,749][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:31,750][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:36,755][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:41,756][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:46,761][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:51,765][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:08:56,771][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:01,781][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:06,784][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:11,786][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:16,792][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:21,794][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:26,799][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:31,801][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:36,807][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:41,815][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:46,817][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:51,821][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:09:56,822][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:01,828][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:06,836][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:11,844][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:16,847][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:21,848][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:26,851][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:31,857][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:36,859][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:41,873][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:46,876][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:51,878][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:10:56,880][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:01,881][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:06,886][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:11,887][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:16,892][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:21,894][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:26,896][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:31,903][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:36,914][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:41,916][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:46,924][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:51,928][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:11:56,929][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:01,931][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:06,945][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:11,947][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:16,950][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:21,953][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:26,962][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:31,963][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:37,122][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:42,136][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:47,144][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:52,146][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:12:57,148][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:02,267][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:07,270][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:12,274][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:17,276][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:22,280][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:27,292][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:32,298][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:37,302][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:42,304][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:47,305][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:52,309][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:13:57,313][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:02,314][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:07,316][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:12,317][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:17,320][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:22,327][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:27,341][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:32,390][  INFO][PID:2313867] Periodic builder liveness probe: alive
[2025-07-23 03:14:32,390][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:37,397][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:42,406][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:47,526][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:52,653][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:14:57,662][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:02,695][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:07,698][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:12,702][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:17,707][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:22,711][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:27,712][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:32,714][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:37,716][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:42,718][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:47,726][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:52,730][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:15:57,739][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:02,741][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:07,744][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:12,760][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:17,766][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:22,768][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:27,770][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:32,773][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:37,774][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:42,776][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:47,778][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:52,784][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:16:57,790][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:02,792][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:07,796][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:12,800][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:17,803][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:22,806][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:27,809][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:32,810][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:37,814][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:42,816][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:47,823][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:52,827][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:17:57,830][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:02,833][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:07,836][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:12,838][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:17,842][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:22,845][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:27,847][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:32,852][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:37,854][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:42,859][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:47,867][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:52,874][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:18:57,879][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:02,880][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:07,882][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:12,883][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:17,887][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:22,892][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:27,893][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:32,895][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:37,902][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:42,906][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:47,908][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:52,916][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:19:57,918][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:02,923][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:07,925][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:12,928][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:17,931][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:22,140][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:20:22,933][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:27,935][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:32,147][WARNING][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:20:32,936][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:38,015][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:42,148][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 2
[2025-07-23 03:20:42,158][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.199 copr-rpmbuild-log
[2025-07-23 03:20:43,027][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:48,033][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:53,110][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:20:58,130][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:03,148][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:08,199][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:13,204][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:18,214][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:23,217][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:28,225][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:33,234][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:38,240][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:42,257][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:21:43,264][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:48,427][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:52,313][WARNING][PID:2313867] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:21:53,466][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:21:58,481][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:02,320][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 3
[2025-07-23 03:22:02,322][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.199 copr-rpmbuild-log
[2025-07-23 03:22:03,526][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:08,539][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:13,580][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:18,674][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:23,689][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:28,700][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:33,705][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:38,706][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:43,709][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:48,716][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:53,724][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:22:58,725][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:02,439][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:23:03,730][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:08,741][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:12,447][WARNING][PID:2313867] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:23:13,746][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:18,765][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:22,448][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 4
[2025-07-23 03:23:22,462][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.199 copr-rpmbuild-log
[2025-07-23 03:23:23,773][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:28,775][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:33,814][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:38,824][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:43,828][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:48,838][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:53,846][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:23:58,854][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:03,863][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:08,879][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:13,881][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:18,901][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:22,550][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:24:23,913][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:28,922][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:32,560][ ERROR][PID:2313867] Giving up for unstable SSH, failures: 1753240832.14751, 1753240912.3139083, 1753240992.4472206, 1753241072.5601182
[2025-07-23 03:24:32,572][ ERROR][PID:2313867] Re-try request for task on 'ResallocHost, ticket_id=4048609, hostname=140.211.168.199, name=copr_osuosl_p09_prod_05748103_20250723_024739, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-23 03:24:32,575][  INFO][PID:2313867] Releasing VM back to pool
[2025-07-23 03:24:32,598][  INFO][PID:2313867] Retry #2 (on other host)
[2025-07-23 03:24:32,610][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:32,621][  INFO][PID:2313867] VM allocation process starts
[2025-07-23 03:24:32,660][  INFO][PID:2313867] Trying to allocate VM: ResallocHost, ticket_id=4049243, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 03:24:35,696][  INFO][PID:2313867] Allocated host ResallocHost, ticket_id=4049243, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05748182_20250723_025216, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-23 03:24:35,697][  INFO][PID:2313867] Allocating ssh connection to builder
[2025-07-23 03:24:35,697][  INFO][PID:2313867] Checking that builder machine is OK
[2025-07-23 03:24:37,426][  INFO][PID:2313867] Running remote command: copr-builder-ready fedora-eln-ppc64le
[2025-07-23 03:24:38,569][  INFO][PID:2313867] Red Hat subscription not needed for fedora-eln-ppc64le
Builder is ready to be used

[2025-07-23 03:24:38,570][  INFO][PID:2313867] Filling build.info file with builder info
[2025-07-23 03:24:38,573][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:38,583][  INFO][PID:2313867] Not re-notifying FE and msg buses for the new host.
[2025-07-23 03:24:38,589][  INFO][PID:2313867] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9306944-fedora-eln-ppc64le --chroot fedora-eln-ppc64le --detached
[2025-07-23 03:24:39,978][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 1
[2025-07-23 03:24:39,986][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-23 03:24:44,995][  INFO][PID:2313867] Periodic builder liveness probe: alive
[2025-07-23 03:24:44,996][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:50,021][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:24:55,022][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:00,024][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:05,030][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:10,034][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:15,038][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:20,045][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:25,047][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:30,048][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:35,054][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:40,055][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:45,056][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:50,059][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:25:55,060][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:00,062][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:05,063][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:10,068][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:15,071][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:20,074][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:25,075][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:30,078][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:35,079][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:40,081][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:45,082][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:50,083][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:26:55,084][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:00,086][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:05,088][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:10,089][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:15,090][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:20,092][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:25,096][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:30,099][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:35,102][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:40,104][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:45,106][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:50,107][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:27:55,109][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:00,110][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:05,112][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:10,113][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:15,115][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:20,116][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:25,118][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:30,119][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:35,131][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:40,132][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:45,134][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:50,135][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:28:55,137][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:00,138][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:05,140][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:10,142][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:15,146][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:20,148][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:25,150][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:30,153][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:35,155][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:40,160][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:45,168][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:50,173][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:29:55,177][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:00,178][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:05,185][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:10,187][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:15,188][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:20,189][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:25,191][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:30,192][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:35,194][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:40,195][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:45,197][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:50,208][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:30:55,209][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:00,211][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:05,212][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:10,213][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:15,218][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:20,221][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:25,225][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:30,226][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:35,229][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:40,231][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:45,234][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:50,236][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:31:55,239][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:00,246][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:05,247][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:10,248][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:15,249][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:20,250][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:25,253][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:30,254][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:35,256][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:40,257][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:45,258][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:50,268][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:32:55,274][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:00,275][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:05,277][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:10,282][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:15,283][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:20,286][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:25,288][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:30,290][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:35,296][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:40,299][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:45,302][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:50,304][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:33:55,305][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:00,308][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:05,310][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:10,312][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:15,316][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:20,317][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:25,319][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:30,322][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:35,324][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:40,327][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:45,348][  INFO][PID:2313867] Periodic builder liveness probe: alive
[2025-07-23 03:34:45,351][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:50,355][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:34:55,357][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:00,359][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:05,363][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:10,367][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:15,368][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:20,371][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:25,374][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:30,376][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:35,381][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:40,382][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:45,384][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:50,391][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:35:55,398][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:00,399][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:05,403][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:10,404][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:15,406][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:20,411][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:25,412][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:30,414][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:35,419][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:40,422][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:45,423][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:50,425][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:36:55,428][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:00,430][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:05,432][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:10,435][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:15,437][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:20,439][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:25,445][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:30,446][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:35,448][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:40,451][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:45,455][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:50,456][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:37:55,457][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:00,458][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:05,461][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:10,464][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:15,466][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:20,469][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:25,470][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:30,473][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:35,475][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:40,481][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:45,482][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:50,483][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:38:55,488][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:00,495][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:05,497][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:10,503][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:15,505][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:20,515][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:25,516][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:30,520][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:35,526][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:40,530][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:45,531][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:50,537][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:39:55,538][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:00,540][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:05,542][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:10,546][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:15,547][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:20,550][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:25,552][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:30,556][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:35,557][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:40,559][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:45,560][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:50,564][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:40:55,566][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:00,568][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:05,569][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:10,573][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:15,574][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:20,575][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:25,577][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:29,140][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:41:30,581][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:35,588][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:39,141][WARNING][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:41:40,590][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:45,591][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:49,142][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 2
[2025-07-23 03:41:49,143][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-23 03:41:50,592][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:41:55,594][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:00,595][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:05,597][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:10,600][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:15,602][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:20,603][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:25,606][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:30,613][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:35,617][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:40,620][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:45,622][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:49,215][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:42:50,626][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:55,628][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:42:59,220][WARNING][PID:2313867] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:43:00,631][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:05,633][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:09,221][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 3
[2025-07-23 03:43:09,235][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-23 03:43:10,634][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:15,637][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:20,639][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:25,644][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:30,649][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:35,650][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:40,652][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:45,654][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:50,657][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:43:55,659][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:00,661][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:05,663][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:09,321][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:44:10,669][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:15,671][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:19,328][WARNING][PID:2313867] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-23 03:44:20,673][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:25,675][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:29,330][  INFO][PID:2313867] Downloading the builder-live.log file, attempt 4
[2025-07-23 03:44:29,332][  INFO][PID:2313867] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-23 03:44:30,679][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:35,681][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:40,684][  INFO][PID:2313867] Checking for cancel request
[2025-07-23 03:44:45,718][  INFO][PID:2313867] Periodic builder liveness probe: dead
[2025-07-23 03:45:29,404][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-23 03:45:39,410][ ERROR][PID:2313867] Giving up for unstable SSH, failures: 1753242099.1411133, 1753242179.2203033, 1753242259.328565, 1753242339.410717
[2025-07-23 03:45:45,800][ ERROR][PID:2313867] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-23 03:46:55,879][ ERROR][PID:2313867] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-23 03:48:05,959][ ERROR][PID:2313867] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-23 03:49:16,050][ ERROR][PID:2313867] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-23 03:49:26,051][ ERROR][PID:2313867] Can't ssh to cancel build.
[2025-07-23 03:49:26,052][ ERROR][PID:2313867] Re-try request for task on 'ResallocHost, ticket_id=4049243, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05748182_20250723_025216, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-07-23 03:49:26,053][  INFO][PID:2313867] Releasing VM back to pool
[2025-07-23 03:49:26,065][ ERROR][PID:2313867] Backend process error: Three host tried without success: {'140.211.168.8', '140.211.168.61', '140.211.168.199'}
[2025-07-23 03:49:26,065][WARNING][PID:2313867] Switching not-finished job state to 'failed'
[2025-07-23 03:49:26,066][  INFO][PID:2313867] Worker failed build, took 1487.4824256896973
[2025-07-23 03:49:26,066][  INFO][PID:2313867] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "rpmsoftwaremanagement",
            "project_name": "dnf-nightly",
            "project_dirname": "dnf-nightly",
            "submitter": "rpmsoftwaremanagement",
            "ended_on": 1753242566.065686,
            "started_on": 1753241078.5832603,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-eln-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9306944-fedora-eln-ppc64le",
            "build_id": 9306944,
            "package_name": "microdnf",
            "package_version": "3.10.1-20250723012110.1.g26c9397",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rpmsoftwaremanagement/dnf-nightly/microdnf",
            "git_hash": "cd40f95006bd643321ae28bdb94eca4be247f6d8",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rpmsoftwaremanagement/dnf-nightly--rpmsoftwaremanagement",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://kojipkgs.fedoraproject.org/repos/eln-build/latest/$basearch",
                    "id": "https_kojipkgs_fedoraproject_org_repos_eln_build_latest_basearch",
                    "name": "Additional repo https_kojipkgs_fedoraproject_org_repos_eln_build_latest_basearch"
                }
            ],
            "background": false,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly",
            "result_dir": "09306944-microdnf",
            "built_packages": "",
            "id": 9306944,
            "mockchain_macros": {
                "copr_username": "rpmsoftwaremanagement",
                "copr_projectname": "dnf-nightly",
                "vendor": "Fedora Project COPR (rpmsoftwaremanagement/dnf-nightly)"
            }
        }
    ]
}
[2025-07-23 03:49:26,106][  INFO][PID:2313867] Sending fedora-messaging bus message in build.end
[2025-07-23 03:49:26,220][  INFO][PID:2313867] Compressing /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/09306944-microdnf/builder-live.log by gzip
[2025-07-23 03:49:26,222][  INFO][PID:2313867] Running command 'gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/09306944-microdnf/builder-live.log' as PID 2699171
[2025-07-23 03:49:26,224][  INFO][PID:2313867] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/09306944-microdnf/builder-live.log)
[2025-07-23 03:49:26,225][  INFO][PID:2313867] Compressing /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/09306944-microdnf/backend.log by gzip
[2025-07-23 03:49:26,226][  INFO][PID:2313867] Running command 'gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly/fedora-eln-ppc64le/09306944-microdnf/backend.log' as PID 2699172