[2025-07-24 06:31:35,803][  INFO][PID:1891628] Marking build as starting
[2025-07-24 06:31:35,930][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:31:35,931][  INFO][PID:1891628] VM allocation process starts
[2025-07-24 06:31:35,947][  INFO][PID:1891628] Trying to allocate VM: ResallocHost, ticket_id=4087453, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-24 06:31:40,948][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:31:41,975][  INFO][PID:1891628] Allocated host ResallocHost, ticket_id=4087453, hostname=140.211.168.94, name=copr_osuosl_p09_prod_05778685_20250724_062523, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-24 06:31:41,976][  INFO][PID:1891628] Allocating ssh connection to builder
[2025-07-24 06:31:41,976][  INFO][PID:1891628] Checking that builder machine is OK
[2025-07-24 06:31:43,486][  INFO][PID:1891628] Running remote command: copr-builder-ready epel-8-ppc64le
[2025-07-24 06:31:44,490][  INFO][PID:1891628] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-07-24 06:31:44,491][  INFO][PID:1891628] Filling build.info file with builder info
[2025-07-24 06:31:44,491][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:31:44,492][  INFO][PID:1891628] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "image-builder",
            "project_dirname": "image-builder",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1753338704.492479,
            "submitted_on": null,
            "status": 3,
            "chroot": "epel-8-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9317361-epel-8-ppc64le",
            "build_id": 9317361,
            "package_name": "koji-image-builder",
            "package_version": "7-1.20250724063109981164.main.3.gaa1e4a5",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/image-builder/koji-image-builder",
            "git_hash": "dd1ed49145589c0074642ae0d5c86b8d0ea9dcc4",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/image-builder--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/image-builder/epel-8-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/image-builder",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/image-builder",
            "result_dir": "09317361-koji-image-builder",
            "built_packages": "",
            "id": 9317361,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "image-builder",
                "vendor": "Fedora Project COPR (@osbuild/image-builder)"
            }
        }
    ]
}
[2025-07-24 06:31:44,532][  INFO][PID:1891628] Sending fedora-messaging bus message in build.start
[2025-07-24 06:31:45,230][  INFO][PID:1891628] Sending fedora-messaging bus message in chroot.start
[2025-07-24 06:31:45,257][  INFO][PID:1891628] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9317361-epel-8-ppc64le --chroot epel-8-ppc64le --detached
[2025-07-24 06:31:46,576][  INFO][PID:1891628] Downloading the builder-live.log file, attempt 1
[2025-07-24 06:31:46,578][  INFO][PID:1891628] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-24 06:31:51,582][  INFO][PID:1891628] Periodic builder liveness probe: alive
[2025-07-24 06:31:51,583][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:31:56,584][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:01,585][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:06,586][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:11,587][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:16,588][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:21,589][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:26,590][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:31,591][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:36,593][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:41,594][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:46,595][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:51,597][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:32:56,598][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:01,599][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:06,600][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:11,601][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:16,602][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:21,604][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:26,612][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:31,614][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:36,615][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:41,616][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:46,617][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:51,619][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:33:56,620][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:01,621][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:06,622][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:11,623][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:16,624][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:21,626][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:26,627][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:31,628][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:36,629][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:41,630][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:46,632][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:51,633][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:34:56,634][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:01,636][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:06,637][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:11,638][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:16,639][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:21,641][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:26,642][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:31,643][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:36,644][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:41,646][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:46,647][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:51,648][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:35:56,649][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:01,652][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:06,658][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:11,659][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:16,660][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:21,661][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:26,662][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:31,663][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:36,664][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:41,666][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:46,667][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:51,668][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:36:56,669][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:01,670][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:06,671][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:11,672][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:16,673][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:21,674][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:26,676][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:31,677][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:36,678][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:41,679][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:46,680][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:51,682][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:37:56,683][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:01,684][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:06,685][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:11,686][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:16,687][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:21,688][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:26,689][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:31,690][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:36,691][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:41,692][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:46,693][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:51,694][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:38:56,695][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:01,696][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:06,697][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:11,698][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:16,699][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:21,700][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:26,701][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:31,702][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:36,703][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:41,704][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:46,706][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:51,707][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:39:56,708][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:01,709][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:06,710][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:11,711][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:16,712][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:21,713][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:26,714][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:31,715][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:36,716][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:41,717][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:46,718][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:51,720][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:40:56,721][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:01,722][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:06,723][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:11,724][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:16,725][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:21,726][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:26,727][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:31,729][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:36,732][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:41,733][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:46,734][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:51,743][  INFO][PID:1891628] Periodic builder liveness probe: alive
[2025-07-24 06:41:51,743][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:41:56,744][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:01,745][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:06,746][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:11,747][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:16,748][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:21,749][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:26,750][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:31,751][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:36,753][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:41,754][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:46,755][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:51,756][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:42:56,758][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:01,759][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:06,760][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:11,761][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:16,762][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:21,763][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:26,764][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:31,767][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:36,768][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:41,771][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:46,773][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:51,776][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:43:56,777][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:01,778][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:06,780][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:11,781][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:16,783][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:21,784][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:26,785][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:31,786][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:36,787][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:41,789][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:46,790][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:51,792][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:44:56,793][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:01,794][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:06,795][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:11,796][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:16,797][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:21,798][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:26,801][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:31,802][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:36,803][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:41,804][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:46,805][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:51,806][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:45:56,808][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:01,809][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:06,810][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:11,811][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:16,812][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:21,814][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:26,815][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:31,816][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:36,817][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:41,818][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:46,819][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:51,822][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:46:56,823][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:01,824][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:06,826][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:11,827][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:16,828][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:21,829][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:26,831][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:31,832][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:36,833][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:41,834][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:46,835][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:51,837][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:47:56,838][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:01,839][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:06,840][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:11,841][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:16,842][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:21,843][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:26,844][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:31,845][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:36,846][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:41,848][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:46,849][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:51,850][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:48:56,852][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:01,853][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:06,854][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:11,855][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:16,856][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:21,859][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:26,861][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:31,862][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:36,864][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:41,865][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:46,866][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:51,873][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:49:56,880][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:01,889][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:06,890][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:11,891][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:16,893][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:21,894][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:26,899][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:31,900][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:36,901][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:41,902][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:46,903][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:51,904][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:50:56,906][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:01,909][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:06,911][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:11,914][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:16,916][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:21,919][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:26,921][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:31,922][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:36,923][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:41,925][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:46,926][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:51,934][  INFO][PID:1891628] Periodic builder liveness probe: alive
[2025-07-24 06:51:51,935][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:51:56,936][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:01,937][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:06,938][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:11,939][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:16,941][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:21,942][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:26,943][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:31,944][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:36,946][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:41,947][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:46,948][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:51,949][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:52:56,951][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:01,953][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:06,955][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:11,957][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:16,958][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:21,959][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:26,960][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:31,962][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:36,963][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:41,965][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:46,966][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:51,968][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:53:56,972][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:01,975][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:06,977][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:11,978][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:16,979][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:21,980][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:26,982][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:31,983][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:36,985][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:41,987][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:46,991][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:51,993][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:54:56,994][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:01,996][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:06,997][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:11,999][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:17,000][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:22,002][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:27,004][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:32,005][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:37,007][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:42,008][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:47,009][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:52,010][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:55:57,014][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:02,018][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:07,019][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:12,020][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:17,025][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:22,031][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:27,032][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:32,033][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:37,034][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:42,035][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:47,036][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:52,037][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:56:57,042][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:02,044][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:07,046][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:12,047][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:17,049][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:22,054][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:27,055][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:32,056][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:37,057][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:42,059][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:47,060][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:52,061][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:57:57,062][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:02,063][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:07,065][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:12,066][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:17,067][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:22,068][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:27,070][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:32,076][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:37,077][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:42,078][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:47,079][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:52,080][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:58:57,082][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:02,083][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:07,084][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:12,085][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:17,087][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:22,088][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:27,089][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:32,090][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:37,092][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:42,093][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:47,094][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:52,095][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 06:59:57,099][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:02,100][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:07,101][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:12,102][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:17,103][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:22,105][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:27,108][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:32,109][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:37,110][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:42,112][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:47,114][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:52,118][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:00:57,120][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:02,121][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:07,122][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:12,123][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:17,124][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:22,126][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:27,127][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:32,131][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:37,133][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:42,135][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:47,136][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:01:52,149][  INFO][PID:1891628] Periodic builder liveness probe: dead
[2025-07-24 07:01:53,142][ ERROR][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-24 07:02:03,143][WARNING][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-24 07:02:13,144][  INFO][PID:1891628] Downloading the builder-live.log file, attempt 2
[2025-07-24 07:02:13,145][  INFO][PID:1891628] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-24 07:02:52,220][ ERROR][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.94 port 22: Connection timed out

[2025-07-24 07:03:13,214][ ERROR][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-24 07:03:23,217][WARNING][PID:1891628] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-24 07:03:33,217][  INFO][PID:1891628] Downloading the builder-live.log file, attempt 3
[2025-07-24 07:03:33,219][  INFO][PID:1891628] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-24 07:04:02,232][ ERROR][PID:1891628] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.94 port 22: Connection timed out

[2025-07-24 07:04:33,287][ ERROR][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-24 07:04:43,292][WARNING][PID:1891628] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-24 07:04:53,293][  INFO][PID:1891628] Downloading the builder-live.log file, attempt 4
[2025-07-24 07:04:53,294][  INFO][PID:1891628] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-24 07:05:12,303][ ERROR][PID:1891628] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.94 port 22: Connection timed out

[2025-07-24 07:05:53,362][ ERROR][PID:1891628] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-24 07:06:03,366][ ERROR][PID:1891628] Giving up for unstable SSH, failures: 1753340523.1432872, 1753340603.21697, 1753340683.2922568, 1753340763.3665805
[2025-07-24 07:06:22,373][ ERROR][PID:1891628] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.94 port 22: Connection timed out

[2025-07-24 07:06:32,374][ ERROR][PID:1891628] Can't ssh to cancel build.
[2025-07-24 07:06:32,375][ ERROR][PID:1891628] Re-try request for task on 'ResallocHost, ticket_id=4087453, hostname=140.211.168.94, name=copr_osuosl_p09_prod_05778685_20250724_062523, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-07-24 07:06:32,376][  INFO][PID:1891628] Releasing VM back to pool
[2025-07-24 07:06:32,395][  INFO][PID:1891628] Retry #1 (on other host)
[2025-07-24 07:06:32,396][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:06:32,396][  INFO][PID:1891628] VM allocation process starts
[2025-07-24 07:06:32,416][  INFO][PID:1891628] Trying to allocate VM: ResallocHost, ticket_id=4087940, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-24 07:06:35,432][  INFO][PID:1891628] Allocated host ResallocHost, ticket_id=4087940, hostname=140.211.168.8, name=copr_osuosl_p09_prod_05779119_20250724_070008, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-24 07:06:35,433][  INFO][PID:1891628] Allocating ssh connection to builder
[2025-07-24 07:06:35,434][  INFO][PID:1891628] Checking that builder machine is OK
[2025-07-24 07:06:36,899][  INFO][PID:1891628] Running remote command: copr-builder-ready epel-8-ppc64le
[2025-07-24 07:06:37,915][  INFO][PID:1891628] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-07-24 07:06:37,917][  INFO][PID:1891628] Filling build.info file with builder info
[2025-07-24 07:06:37,922][  INFO][PID:1891628] Checking for cancel request
[2025-07-24 07:06:37,925][  INFO][PID:1891628] Not re-notifying FE and msg buses for the new host.
[2025-07-24 07:06:37,927][  INFO][PID:1891628] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9317361-epel-8-ppc64le --chroot epel-8-ppc64le --detached
[2025-07-24 07:06:39,260][  INFO][PID:1891628] Downloading the builder-live.log file, attempt 1
[2025-07-24 07:06:39,267][  INFO][PID:1891628] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.8 copr-rpmbuild-log
[2025-07-24 07:06:48,419][  INFO][PID:1891628] Cancel request succeeded
out:
success
err:
Warning: Permanently added '140.211.168.8' (ED25519) to the list of known hosts.

[2025-07-24 07:06:48,419][  INFO][PID:1891628] Downloading results from builder
[2025-07-24 07:06:48,420][  INFO][PID:1891628] rsyncing of mockbuilder@140.211.168.8:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder started
[2025-07-24 07:06:48,420][  INFO][PID:1891628] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.8:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/ &> /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/build-09317361.rsync.log
[2025-07-24 07:06:49,971][  INFO][PID:1891628] rsyncing finished.
[2025-07-24 07:06:49,972][  INFO][PID:1891628] Releasing VM back to pool
[2025-07-24 07:06:49,981][  INFO][PID:1891628] Searching for 'success' file in resultdir
[2025-07-24 07:06:49,982][ ERROR][PID:1891628] Build failed: Backend process error: No success file => build failure
[2025-07-24 07:06:49,982][  INFO][PID:1891628] Finished build: id=9317361 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/@osbuild/image-builder chroot=epel-8-ppc64le 
[2025-07-24 07:06:49,985][ ERROR][PID:1891628] Backend process error: No success file => build failure
[2025-07-24 07:06:49,986][  INFO][PID:1891628] Worker failed build, took 12.061087846755981
[2025-07-24 07:06:49,986][  INFO][PID:1891628] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "image-builder",
            "project_dirname": "image-builder",
            "submitter": "packit",
            "ended_on": 1753340809.986248,
            "started_on": 1753340797.9251602,
            "submitted_on": null,
            "status": 0,
            "chroot": "epel-8-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9317361-epel-8-ppc64le",
            "build_id": 9317361,
            "package_name": "koji-image-builder",
            "package_version": "7-1.20250724063109981164.main.3.gaa1e4a5",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/image-builder/koji-image-builder",
            "git_hash": "dd1ed49145589c0074642ae0d5c86b8d0ea9dcc4",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/image-builder--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/image-builder/epel-8-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/image-builder",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/image-builder",
            "result_dir": "09317361-koji-image-builder",
            "built_packages": "",
            "id": 9317361,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "image-builder",
                "vendor": "Fedora Project COPR (@osbuild/image-builder)"
            }
        }
    ]
}
[2025-07-24 07:06:50,029][  INFO][PID:1891628] Sending fedora-messaging bus message in build.end
[2025-07-24 07:06:50,058][  INFO][PID:1891628] Compressing /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/builder-live.log by gzip
[2025-07-24 07:06:50,059][  INFO][PID:1891628] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/builder-live.log' as PID 2098019
[2025-07-24 07:06:50,061][  INFO][PID:1891628] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/builder-live.log)
[2025-07-24 07:06:50,061][  INFO][PID:1891628] Compressing /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/backend.log by gzip
[2025-07-24 07:06:50,062][  INFO][PID:1891628] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/image-builder/epel-8-ppc64le/09317361-koji-image-builder/backend.log' as PID 2098021