[2026-01-14 02:35:12,803][  INFO][PID:3231436] Marking build as starting
[2026-01-14 02:35:12,865][  INFO][PID:3231436] Checking that /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 02:35:12,877][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:12,878][  INFO][PID:3231436] VM allocation process starts
[2026-01-14 02:35:12,912][  INFO][PID:3231436] Trying to allocate VM: ResallocHost, ticket_id=803540, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:35:17,917][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:22,918][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:27,919][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:32,921][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:37,924][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:42,925][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:47,926][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:52,927][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:35:57,928][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:02,928][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:07,930][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:12,931][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:17,932][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:22,942][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:27,943][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:32,944][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:37,945][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:42,946][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:47,947][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:52,948][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:36:57,949][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:02,950][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:07,951][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:12,952][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:17,956][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:22,959][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:27,961][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:32,962][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:37,965][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:42,967][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:47,967][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:52,968][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:37:57,969][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:02,970][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:07,971][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:12,972][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:17,973][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:22,974][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:27,975][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:30,997][  INFO][PID:3231436] Allocated host ResallocHost, ticket_id=803540, hostname=2620:52:6:1161:dead:beef:cafe:c193, name=vmhost_p09_04_prod_01216755_20260114_023311, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:38:30,998][  INFO][PID:3231436] Allocating ssh connection to builder
[2026-01-14 02:38:30,998][  INFO][PID:3231436] Checking that builder machine is OK
[2026-01-14 02:38:31,869][  INFO][PID:3231436] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-14 02:38:32,399][  INFO][PID:3231436] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2026-01-14 02:38:32,400][  INFO][PID:3231436] Filling build.info file with builder info
[2026-01-14 02:38:32,401][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:38:32,402][  INFO][PID:3231436] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ryanabx",
            "project_name": "cosmic-epoch-tagged",
            "project_dirname": "cosmic-epoch-tagged",
            "submitter": "ryanabx",
            "ended_on": null,
            "started_on": 1768358312.401905,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10005605-fedora-rawhide-ppc64le",
            "build_id": 10005605,
            "package_name": "cosmic-applets",
            "package_version": "1.0.2-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ryanabx/cosmic-epoch-tagged/cosmic-applets",
            "git_hash": "84d2d388c7078b486df8466c80e0521054136877",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ryanabx/cosmic-epoch-tagged--ryanabx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": true,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged",
            "result_dir": "10005605-cosmic-applets",
            "built_packages": "",
            "id": 10005605,
            "mockchain_macros": {
                "copr_username": "ryanabx",
                "copr_projectname": "cosmic-epoch-tagged",
                "vendor": "Fedora Project COPR (ryanabx/cosmic-epoch-tagged)"
            }
        }
    ]
}
[2026-01-14 02:38:32,451][  INFO][PID:3231436] Sending fedora-messaging bus message in build.start
[2026-01-14 02:38:33,019][  INFO][PID:3231436] Sending fedora-messaging bus message in chroot.start
[2026-01-14 02:38:33,048][  INFO][PID:3231436] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10005605-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2026-01-14 02:39:33,119][ ERROR][PID:3231436] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c193 port 22: Connection timed out

[2026-01-14 02:39:43,120][ ERROR][PID:3231436] Re-try request for task on 'ResallocHost, ticket_id=803540, hostname=2620:52:6:1161:dead:beef:cafe:c193, name=vmhost_p09_04_prod_01216755_20260114_023311, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-14 02:39:43,120][  INFO][PID:3231436] Releasing VM back to pool
[2026-01-14 02:39:43,132][  INFO][PID:3231436] Retry #1 (on other host)
[2026-01-14 02:39:43,133][  INFO][PID:3231436] Checking that /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 02:39:43,133][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:39:43,134][  INFO][PID:3231436] VM allocation process starts
[2026-01-14 02:39:43,147][  INFO][PID:3231436] Trying to allocate VM: ResallocHost, ticket_id=803650, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:39:48,149][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:39:53,150][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:39:58,151][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:03,152][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:08,153][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:13,154][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:18,155][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:23,156][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:28,157][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:33,158][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:38,159][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:43,160][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:48,161][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:53,162][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:40:58,164][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:03,165][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:08,174][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:13,179][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:18,180][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:23,181][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:28,182][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:33,183][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:38,184][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:43,185][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:48,187][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:53,188][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:41:58,189][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:03,190][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:08,191][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:13,192][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:18,193][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:23,194][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:28,195][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:33,197][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:38,198][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:43,199][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:48,200][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:53,201][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:42:58,202][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:03,204][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:08,205][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:13,206][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:18,207][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:23,208][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:28,209][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:33,210][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:38,211][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:43,212][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:48,213][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:53,214][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:43:58,215][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:03,216][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:08,217][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:13,218][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:18,219][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:23,220][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:28,221][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:33,221][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:38,223][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:43,224][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:48,225][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:53,226][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:44:58,226][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:03,227][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:08,228][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:13,229][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:18,230][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:23,231][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:28,232][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:33,233][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:38,234][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:43,235][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:48,236][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:53,237][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:45:58,238][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:03,278][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:08,279][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:13,305][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:18,306][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:23,308][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:28,309][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:33,310][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:38,475][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:43,476][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:48,519][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:53,519][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:46:58,520][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:03,521][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:08,522][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:13,523][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:18,670][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:23,671][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:28,673][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:33,673][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:38,674][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:43,735][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:48,738][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:53,739][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:47:58,740][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:03,740][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:08,741][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:13,743][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:18,784][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:23,785][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:28,912][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:33,926][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:38,928][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:43,928][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:48,929][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:53,930][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:48:58,931][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:49:01,287][  INFO][PID:3231436] Allocated host ResallocHost, ticket_id=803650, hostname=2620:52:6:1161:dead:beef:cafe:c18e, name=vmhost_p09_04_prod_01216821_20260114_023858, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:49:01,287][  INFO][PID:3231436] Allocating ssh connection to builder
[2026-01-14 02:49:01,287][  INFO][PID:3231436] Checking that builder machine is OK
[2026-01-14 02:50:01,355][ ERROR][PID:3231436] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18e port 22: Connection timed out

[2026-01-14 02:50:11,375][ ERROR][PID:3231436] Re-try request for task on 'ResallocHost, ticket_id=803650, hostname=2620:52:6:1161:dead:beef:cafe:c18e, name=vmhost_p09_04_prod_01216821_20260114_023858, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-14 02:50:11,375][  INFO][PID:3231436] Releasing VM back to pool
[2026-01-14 02:50:11,386][  INFO][PID:3231436] Retry #2 (on other host)
[2026-01-14 02:50:11,386][  INFO][PID:3231436] Checking that /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-14 02:50:11,386][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:11,387][  INFO][PID:3231436] VM allocation process starts
[2026-01-14 02:50:11,397][  INFO][PID:3231436] Trying to allocate VM: ResallocHost, ticket_id=803722, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:50:16,398][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:21,399][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:26,400][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:31,401][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:36,402][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:41,403][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:46,404][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:51,405][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:50:56,407][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:01,408][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:06,409][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:11,410][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:16,411][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:21,412][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:26,413][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:31,414][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:36,415][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:41,416][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:46,417][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:51,418][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:51:56,419][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:01,420][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:06,421][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:11,422][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:16,423][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:21,424][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:26,425][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:31,426][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:36,427][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:41,428][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:46,429][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:51,429][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:52:56,430][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:01,432][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:06,432][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:11,433][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:16,434][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:21,454][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:26,455][  INFO][PID:3231436] Checking for cancel request
[2026-01-14 02:53:29,459][  INFO][PID:3231436] Allocated host ResallocHost, ticket_id=803722, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01216885_20260114_024337, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-14 02:53:30,173][  INFO][PID:3231436] Allocating ssh connection to builder
[2026-01-14 02:53:30,174][  INFO][PID:3231436] Checking that builder machine is OK
[2026-01-14 02:54:30,243][ ERROR][PID:3231436] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18d port 22: Connection timed out

[2026-01-14 02:54:40,250][ ERROR][PID:3231436] Re-try request for task on 'ResallocHost, ticket_id=803722, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01216885_20260114_024337, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-14 02:54:40,250][  INFO][PID:3231436] Releasing VM back to pool
[2026-01-14 02:54:40,258][ ERROR][PID:3231436] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c18e', '2620:52:6:1161:dead:beef:cafe:c193', '2620:52:6:1161:dead:beef:cafe:c18d'}
[2026-01-14 02:54:40,259][WARNING][PID:3231436] Switching not-finished job state to 'failed'
[2026-01-14 02:54:40,259][  INFO][PID:3231436] Worker failed build, took 967.8573434352875
[2026-01-14 02:54:40,260][  INFO][PID:3231436] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ryanabx",
            "project_name": "cosmic-epoch-tagged",
            "project_dirname": "cosmic-epoch-tagged",
            "submitter": "ryanabx",
            "ended_on": 1768359280.2592485,
            "started_on": 1768358312.401905,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10005605-fedora-rawhide-ppc64le",
            "build_id": 10005605,
            "package_name": "cosmic-applets",
            "package_version": "1.0.2-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ryanabx/cosmic-epoch-tagged/cosmic-applets",
            "git_hash": "84d2d388c7078b486df8466c80e0521054136877",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ryanabx/cosmic-epoch-tagged--ryanabx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": true,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ryanabx/cosmic-epoch-tagged",
            "result_dir": "10005605-cosmic-applets",
            "built_packages": "",
            "id": 10005605,
            "mockchain_macros": {
                "copr_username": "ryanabx",
                "copr_projectname": "cosmic-epoch-tagged",
                "vendor": "Fedora Project COPR (ryanabx/cosmic-epoch-tagged)"
            }
        }
    ]
}
[2026-01-14 02:54:40,308][  INFO][PID:3231436] Sending fedora-messaging bus message in build.end
[2026-01-14 02:54:40,335][  INFO][PID:3231436] Compressing /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/builder-live.log by gzip
[2026-01-14 02:54:40,336][  INFO][PID:3231436] Running command 'gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/builder-live.log' as PID 3327128
[2026-01-14 02:54:40,337][  INFO][PID:3231436] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/builder-live.log: No such file or directory


[2026-01-14 02:54:40,338][ ERROR][PID:3231436] Unable to compress file /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/builder-live.log
[2026-01-14 02:54:40,338][  INFO][PID:3231436] Compressing /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/backend.log by gzip
[2026-01-14 02:54:40,339][  INFO][PID:3231436] Running command 'gzip /var/lib/copr/public_html/results/ryanabx/cosmic-epoch-tagged/fedora-rawhide-ppc64le/10005605-cosmic-applets/backend.log' as PID 3327129