[2025-07-21 09:32:03,427][  INFO][PID:242747] Marking build as starting
[2025-07-21 09:32:03,624][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:03,626][  INFO][PID:242747] VM allocation process starts
[2025-07-21 09:32:03,686][  INFO][PID:242747] Trying to allocate VM: ResallocHost, ticket_id=4007764, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-21 09:32:08,688][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:13,689][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:18,698][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:23,704][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:28,706][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:31,957][  INFO][PID:242747] Allocated host ResallocHost, ticket_id=4007764, hostname=140.211.168.48, name=copr_osuosl_p09_prod_05628938_20250719_012107, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-21 09:32:31,957][  INFO][PID:242747] Allocating ssh connection to builder
[2025-07-21 09:32:31,962][  INFO][PID:242747] Checking that builder machine is OK
[2025-07-21 09:32:33,545][  INFO][PID:242747] Running remote command: copr-builder-ready fedora-42-ppc64le
[2025-07-21 09:32:34,592][  INFO][PID:242747] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2025-07-21 09:32:34,593][  INFO][PID:242747] Filling build.info file with builder info
[2025-07-21 09:32:34,594][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:34,600][  INFO][PID:242747] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": null,
            "started_on": 1753090354.599875,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9294035-fedora-42-ppc64le",
            "build_id": 9294035,
            "package_name": "ipython",
            "package_version": "8.37.0-6",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/ipython",
            "git_hash": "86fef353ae3d8ad482aeec46c1fdf6c4c03ffd22",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09294035-ipython",
            "built_packages": "",
            "id": 9294035,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-21 09:32:34,666][  INFO][PID:242747] Sending fedora-messaging bus message in build.start
[2025-07-21 09:32:35,547][  INFO][PID:242747] Sending fedora-messaging bus message in chroot.start
[2025-07-21 09:32:35,572][  INFO][PID:242747] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9294035-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2025-07-21 09:32:36,965][  INFO][PID:242747] Downloading the builder-live.log file, attempt 1
[2025-07-21 09:32:36,974][  INFO][PID:242747] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.48 copr-rpmbuild-log
[2025-07-21 09:32:41,979][  INFO][PID:242747] Periodic builder liveness probe: alive
[2025-07-21 09:32:41,980][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:46,981][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:51,984][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:32:56,993][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:01,994][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:06,998][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:12,001][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:17,005][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:22,009][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:27,011][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:32,012][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:37,018][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:42,025][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:47,031][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:52,032][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:33:57,033][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:02,039][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:07,041][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:12,043][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:17,045][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:22,048][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:27,052][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:32,058][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:37,064][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:42,065][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:47,070][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:52,072][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:34:57,080][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:02,081][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:07,084][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:12,088][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:17,094][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:22,096][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:27,099][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:32,100][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:37,102][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:42,104][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:47,108][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:52,111][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:35:57,114][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:02,115][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:07,116][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:12,121][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:17,123][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:22,124][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:27,129][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:32,134][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:37,135][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:42,136][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:47,138][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:52,143][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:36:57,144][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:02,146][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:07,152][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:12,153][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:17,154][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:22,156][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:27,158][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:32,159][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:37,162][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:42,165][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:47,168][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:52,171][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:37:57,173][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:02,177][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:07,181][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:12,183][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:17,186][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:22,188][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:27,190][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:32,194][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:37,198][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:42,199][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:47,201][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:52,204][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:38:57,206][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:02,208][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:07,213][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:12,215][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:17,218][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:22,221][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:27,226][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:32,227][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:37,228][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:42,229][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:47,231][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:52,233][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:39:57,237][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:02,238][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:07,240][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:12,243][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:17,246][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:22,250][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:27,255][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:32,259][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:37,264][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:42,267][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:47,269][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:52,270][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:40:57,277][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:02,279][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:07,284][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:12,289][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:17,293][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:22,295][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:27,297][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:32,302][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:37,312][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:42,314][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:47,316][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:52,319][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:41:57,321][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:02,323][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:07,326][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:12,329][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:17,331][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:22,336][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:27,338][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:32,342][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:37,349][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:42,396][  INFO][PID:242747] Periodic builder liveness probe: alive
[2025-07-21 09:42:42,397][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:47,404][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:52,405][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:42:57,410][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:02,412][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:07,417][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:12,424][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:17,425][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:22,426][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:27,429][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:32,437][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:37,441][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:42,443][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:47,445][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:52,457][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:43:57,463][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:02,465][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:07,467][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:12,472][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:17,479][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:22,483][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:27,487][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:32,493][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:37,503][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:42,505][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:47,506][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:52,512][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:44:57,514][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:02,516][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:07,517][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:12,523][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:17,525][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:22,527][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:27,529][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:32,531][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:37,539][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:42,540][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:47,544][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:52,546][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:45:57,554][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:02,561][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:07,564][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:12,565][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:17,569][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:22,572][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:27,575][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:32,577][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:37,580][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:42,583][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:47,590][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:52,592][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:46:57,594][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:02,602][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:07,604][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:12,608][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:17,611][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:22,642][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:27,644][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:32,645][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:37,646][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:42,656][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:47,658][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:52,659][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:47:57,662][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:02,666][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:07,684][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:12,701][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:17,707][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:22,708][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:27,711][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:32,715][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:37,719][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:42,732][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:47,734][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:52,735][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:48:57,737][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:02,739][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:07,743][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:12,748][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:17,751][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:22,752][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:28,434][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:33,511][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:39,817][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:45,125][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:50,154][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:49:55,228][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:00,232][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:06,171][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:11,519][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:16,716][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:21,938][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:26,953][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:31,958][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:36,961][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:41,964][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:46,968][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:52,037][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:50:57,085][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:02,091][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:07,342][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:12,344][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:17,345][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:22,347][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:27,352][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:32,361][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:37,369][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:42,381][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:47,395][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:52,397][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:51:57,403][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:02,408][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:07,411][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:12,413][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:17,432][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:22,433][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:27,437][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:32,441][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:37,446][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:42,478][  INFO][PID:242747] Periodic builder liveness probe: alive
[2025-07-21 09:52:42,483][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:47,487][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:52,492][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:52:57,506][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:02,519][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:07,531][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:12,547][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:17,549][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:22,551][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:27,556][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:32,560][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:37,566][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:42,576][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:47,579][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:52,594][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:53:57,601][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:02,614][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:07,616][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:12,620][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:17,623][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:22,637][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:27,648][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:32,658][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:37,663][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:42,670][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:47,683][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:52,709][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:54:57,712][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:02,721][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:07,728][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:12,735][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:17,739][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:22,744][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:27,765][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:32,774][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:37,775][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:42,781][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:47,789][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:52,797][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:55:57,806][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:02,810][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:07,815][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:12,817][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:17,834][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:22,839][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:27,847][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:32,863][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:37,868][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:42,878][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:47,881][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:52,894][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:56:57,896][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:02,902][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:07,913][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:12,924][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:17,930][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:22,941][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:27,952][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:32,965][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:37,991][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:42,994][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:48,017][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:53,020][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:57:58,033][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:03,073][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:08,075][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:13,084][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:18,092][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:23,093][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:28,103][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:33,117][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:38,129][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:43,131][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:48,149][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:53,150][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:58:58,162][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:03,167][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:08,178][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:13,186][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:18,189][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:23,193][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:28,202][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:33,210][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:38,217][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:43,224][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:48,230][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:53,547][  INFO][PID:242747] Checking for cancel request
[2025-07-21 09:59:58,555][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:03,563][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:08,566][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:13,576][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:18,592][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:23,596][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:28,600][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:33,606][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:38,608][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:43,616][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:48,621][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:53,625][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:00:58,633][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:03,646][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:08,648][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:13,651][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:18,654][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:23,659][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:28,660][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:33,665][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:38,667][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:43,672][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:48,674][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:53,679][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:01:58,684][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:03,685][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:08,697][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:13,702][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:18,705][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:23,707][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:28,714][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:33,715][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:38,719][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:02:43,748][  INFO][PID:242747] Periodic builder liveness probe: dead
[2025-07-21 10:03:43,858][ ERROR][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.48 port 22: Connection timed out

[2025-07-21 10:04:04,143][ ERROR][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-21 10:04:14,144][WARNING][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-21 10:04:24,145][  INFO][PID:242747] Downloading the builder-live.log file, attempt 2
[2025-07-21 10:04:24,147][  INFO][PID:242747] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.48 copr-rpmbuild-log
[2025-07-21 10:04:53,949][ ERROR][PID:242747] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.48 port 22: Connection timed out

[2025-07-21 10:05:24,223][ ERROR][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-21 10:05:34,228][WARNING][PID:242747] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-21 10:05:44,229][  INFO][PID:242747] Downloading the builder-live.log file, attempt 3
[2025-07-21 10:05:44,232][  INFO][PID:242747] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.48 copr-rpmbuild-log
[2025-07-21 10:06:04,023][ ERROR][PID:242747] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.48 port 22: Connection timed out

[2025-07-21 10:06:44,335][ ERROR][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-21 10:06:54,352][WARNING][PID:242747] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-21 10:07:04,356][  INFO][PID:242747] Downloading the builder-live.log file, attempt 4
[2025-07-21 10:07:04,360][  INFO][PID:242747] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.48 copr-rpmbuild-log
[2025-07-21 10:07:14,129][ ERROR][PID:242747] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.48 port 22: Connection timed out

[2025-07-21 10:07:24,133][ ERROR][PID:242747] Can't ssh to cancel build.
[2025-07-21 10:08:04,453][ ERROR][PID:242747] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-21 10:08:14,461][ ERROR][PID:242747] Giving up for unstable SSH, failures: 1753092254.1446924, 1753092334.2285864, 1753092414.352126, 1753092494.461106
[2025-07-21 10:08:14,463][ ERROR][PID:242747] Re-try request for task on 'ResallocHost, ticket_id=4007764, hostname=140.211.168.48, name=copr_osuosl_p09_prod_05628938_20250719_012107, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-07-21 10:08:14,465][  INFO][PID:242747] Releasing VM back to pool
[2025-07-21 10:08:14,499][  INFO][PID:242747] Retry #1 (on other host)
[2025-07-21 10:08:14,499][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:08:14,507][  INFO][PID:242747] VM allocation process starts
[2025-07-21 10:08:14,533][  INFO][PID:242747] Trying to allocate VM: ResallocHost, ticket_id=4010011, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-21 10:08:19,540][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:08:24,544][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:08:29,560][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:08:32,669][  INFO][PID:242747] Allocated host ResallocHost, ticket_id=4010011, hostname=140.211.168.156, name=copr_osuosl_p09_prod_05693526_20250721_100159, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-21 10:08:32,670][  INFO][PID:242747] Allocating ssh connection to builder
[2025-07-21 10:08:32,672][  INFO][PID:242747] Checking that builder machine is OK
[2025-07-21 10:08:33,673][  INFO][PID:242747] Running remote command: copr-builder-ready fedora-42-ppc64le
[2025-07-21 10:08:34,726][  INFO][PID:242747] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2025-07-21 10:08:34,726][  INFO][PID:242747] Filling build.info file with builder info
[2025-07-21 10:08:34,732][  INFO][PID:242747] Checking for cancel request
[2025-07-21 10:08:34,734][  INFO][PID:242747] Not re-notifying FE and msg buses for the new host.
[2025-07-21 10:08:34,736][  INFO][PID:242747] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9294035-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2025-07-21 10:08:35,989][  INFO][PID:242747] Downloading the builder-live.log file, attempt 1
[2025-07-21 10:08:35,994][  INFO][PID:242747] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.156 copr-rpmbuild-log
[2025-07-21 10:08:45,214][  INFO][PID:242747] Cancel request succeeded
out:
success
err:
Warning: Permanently added '140.211.168.156' (ED25519) to the list of known hosts.

[2025-07-21 10:08:45,215][  INFO][PID:242747] Downloading results from builder
[2025-07-21 10:08:45,216][  INFO][PID:242747] rsyncing of mockbuilder@140.211.168.156:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython started
[2025-07-21 10:08:45,221][  INFO][PID:242747] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.156:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/ &> /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/build-09294035.rsync.log
[2025-07-21 10:08:46,932][  INFO][PID:242747] rsyncing finished.
[2025-07-21 10:08:46,944][  INFO][PID:242747] Releasing VM back to pool
[2025-07-21 10:08:47,025][  INFO][PID:242747] Searching for 'success' file in resultdir
[2025-07-21 10:08:47,035][ ERROR][PID:242747] Build failed: Backend process error: No success file => build failure
[2025-07-21 10:08:47,043][  INFO][PID:242747] Finished build: id=9294035 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/openblas-0.3.23 chroot=fedora-42-ppc64le 
[2025-07-21 10:08:47,061][ ERROR][PID:242747] Backend process error: No success file => build failure
[2025-07-21 10:08:47,062][  INFO][PID:242747] Worker failed build, took 12.327306747436523
[2025-07-21 10:08:47,062][  INFO][PID:242747] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": 1753092527.0621023,
            "started_on": 1753092514.7347956,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9294035-fedora-42-ppc64le",
            "build_id": 9294035,
            "package_name": "ipython",
            "package_version": "8.37.0-6",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/ipython",
            "git_hash": "86fef353ae3d8ad482aeec46c1fdf6c4c03ffd22",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09294035-ipython",
            "built_packages": "",
            "id": 9294035,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-21 10:08:47,122][  INFO][PID:242747] Sending fedora-messaging bus message in build.end
[2025-07-21 10:08:47,239][  INFO][PID:242747] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/builder-live.log by gzip
[2025-07-21 10:08:47,250][  INFO][PID:242747] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/builder-live.log' as PID 686749
[2025-07-21 10:08:47,261][  INFO][PID:242747] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/builder-live.log)
[2025-07-21 10:08:47,267][  INFO][PID:242747] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/backend.log by gzip
[2025-07-21 10:08:47,274][  INFO][PID:242747] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-42-ppc64le/09294035-ipython/backend.log' as PID 686752