[2025-12-11 13:33:43,023][ INFO][PID:2552227] Marking build as starting [2025-12-11 13:33:43,064][ INFO][PID:2552227] Checking that /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/repodata/repomd.xml exists [2025-12-11 13:33:43,066][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:33:43,066][ INFO][PID:2552227] VM allocation process starts [2025-12-11 13:33:43,090][ INFO][PID:2552227] Trying to allocate VM: ResallocHost, ticket_id=283778, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-11 13:33:48,092][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:33:49,127][ INFO][PID:2552227] Allocated host ResallocHost, ticket_id=283778, hostname=54.162.232.229, name=aws_x86_64_spot_prod_00346646_20251211_132910, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-11 13:33:49,128][ INFO][PID:2552227] Allocating ssh connection to builder [2025-12-11 13:33:49,129][ INFO][PID:2552227] Checking that builder machine is OK [2025-12-11 13:33:49,478][ INFO][PID:2552227] Running remote command: copr-builder-ready centos-stream-10-x86_64 [2025-12-11 13:33:49,758][ INFO][PID:2552227] Red Hat subscription not needed for centos-stream-10-x86_64 Builder is ready to be used [2025-12-11 13:33:49,760][ INFO][PID:2552227] Filling build.info file with builder info [2025-12-11 13:33:49,762][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:33:49,772][ INFO][PID:2552227] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "osloup", "project_name": "docbook5_style_xsl_license_and_jar_build", "project_dirname": "docbook5_style_xsl_license_and_jar_build", "submitter": "https://src.fedoraproject.org/user/jakub", "ended_on": null, "started_on": 1765460029.7723253, "submitted_on": null, "status": 3, "chroot": "centos-stream-10-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9899285-centos-stream-10-x86_64", "build_id": 9899285, "package_name": "gcc", "package_version": "15.2.1-5", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/osloup/docbook5_style_xsl_license_and_jar_build/gcc", "git_hash": "b4ce49277f8b538f2550e01d3179102adad1765a", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "osloup/docbook5_style_xsl_license_and_jar_build--https://src.fedoraproject.org/user/jakub", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/osloup/docbook5_style_xsl_license_and_jar_build", "result_dir": "09899285-gcc", "built_packages": "", "id": 9899285, "mockchain_macros": { "copr_username": "osloup", "copr_projectname": "docbook5_style_xsl_license_and_jar_build", "vendor": "Fedora Project COPR (osloup/docbook5_style_xsl_license_and_jar_build)" } } ] } [2025-12-11 13:33:49,897][ INFO][PID:2552227] Sending fedora-messaging bus message in build.start [2025-12-11 13:33:50,921][ INFO][PID:2552227] Sending fedora-messaging bus message in chroot.start [2025-12-11 13:33:50,955][ INFO][PID:2552227] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9899285-centos-stream-10-x86_64 --chroot centos-stream-10-x86_64 --detached [2025-12-11 13:33:51,278][ INFO][PID:2552227] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.162.232.229' (ED25519) to the list of known hosts. [2025-12-11 13:33:51,279][ INFO][PID:2552227] Downloading the builder-live.log file, attempt 1 [2025-12-11 13:33:51,289][ INFO][PID:2552227] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.232.229 copr-rpmbuild-log [2025-12-11 13:33:56,326][ INFO][PID:2552227] Periodic builder liveness probe: alive [2025-12-11 13:33:56,327][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:01,335][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:06,336][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:11,338][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:16,339][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:21,340][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:26,342][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:31,343][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:36,345][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:41,347][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:45,125][ ERROR][PID:2552227] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-11 13:34:46,348][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:51,349][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:34:55,129][WARNING][PID:2552227] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-11 13:34:56,352][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:01,354][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:05,130][ INFO][PID:2552227] Downloading the builder-live.log file, attempt 2 [2025-12-11 13:35:05,132][ INFO][PID:2552227] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.232.229 copr-rpmbuild-log [2025-12-11 13:35:06,355][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:11,357][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:16,358][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:21,359][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:26,360][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:31,362][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:36,363][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:41,364][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:46,366][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:51,369][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:35:56,370][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:01,372][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:05,208][ ERROR][PID:2552227] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-11 13:36:06,374][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:11,375][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:15,214][WARNING][PID:2552227] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-11 13:36:16,376][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:21,377][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:25,215][ INFO][PID:2552227] Downloading the builder-live.log file, attempt 3 [2025-12-11 13:36:25,216][ INFO][PID:2552227] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.232.229 copr-rpmbuild-log [2025-12-11 13:36:26,378][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:31,379][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:36,380][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:41,381][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:46,382][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:51,383][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:36:56,384][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:01,385][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:06,387][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:11,388][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:16,389][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:21,390][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:25,273][ ERROR][PID:2552227] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-11 13:37:26,390][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:31,391][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:35,275][WARNING][PID:2552227] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-11 13:37:36,392][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:41,393][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:45,275][ INFO][PID:2552227] Downloading the builder-live.log file, attempt 4 [2025-12-11 13:37:45,277][ INFO][PID:2552227] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.232.229 copr-rpmbuild-log [2025-12-11 13:37:46,394][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:51,395][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:37:56,396][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:01,397][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:06,398][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:11,399][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:16,400][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:21,401][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:26,402][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:31,403][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:36,404][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:41,405][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:45,339][ ERROR][PID:2552227] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-11 13:38:46,406][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:51,407][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:55,342][ ERROR][PID:2552227] Giving up for unstable SSH, failures: 1765460095.129704, 1765460175.2144783, 1765460255.275209, 1765460335.34249 [2025-12-11 13:38:55,343][ ERROR][PID:2552227] Re-try request for task on 'ResallocHost, ticket_id=283778, hostname=54.162.232.229, name=aws_x86_64_spot_prod_00346646_20251211_132910, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-11 13:38:55,343][ INFO][PID:2552227] Releasing VM back to pool [2025-12-11 13:38:55,354][ INFO][PID:2552227] Retry #1 (on other host) [2025-12-11 13:38:55,355][ INFO][PID:2552227] Checking that /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/repodata/repomd.xml exists [2025-12-11 13:38:55,355][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:38:55,356][ INFO][PID:2552227] VM allocation process starts [2025-12-11 13:38:55,366][ INFO][PID:2552227] Trying to allocate VM: ResallocHost, ticket_id=283811, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-11 13:39:00,367][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:01,383][ INFO][PID:2552227] Allocated host ResallocHost, ticket_id=283811, hostname=34.200.241.165, name=aws_x86_64_normalreserved_prod_00346743_20251211_133247, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-11 13:39:01,384][ INFO][PID:2552227] Allocating ssh connection to builder [2025-12-11 13:39:01,384][ INFO][PID:2552227] Checking that builder machine is OK [2025-12-11 13:39:01,723][ INFO][PID:2552227] Running remote command: copr-builder-ready centos-stream-10-x86_64 [2025-12-11 13:39:01,922][ INFO][PID:2552227] Red Hat subscription not needed for centos-stream-10-x86_64 Builder is ready to be used [2025-12-11 13:39:01,923][ INFO][PID:2552227] Filling build.info file with builder info [2025-12-11 13:39:01,926][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:01,927][ INFO][PID:2552227] Not re-notifying FE and msg buses for the new host. [2025-12-11 13:39:01,927][ INFO][PID:2552227] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9899285-centos-stream-10-x86_64 --chroot centos-stream-10-x86_64 --detached [2025-12-11 13:39:02,251][ INFO][PID:2552227] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '34.200.241.165' (ED25519) to the list of known hosts. [2025-12-11 13:39:02,252][ INFO][PID:2552227] Downloading the builder-live.log file, attempt 1 [2025-12-11 13:39:02,253][ INFO][PID:2552227] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.200.241.165 copr-rpmbuild-log [2025-12-11 13:39:07,252][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:12,253][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:17,254][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:22,255][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:27,256][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:32,257][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:37,258][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:42,259][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:47,260][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:52,261][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:39:57,262][ INFO][PID:2552227] Checking for cancel request [2025-12-11 13:40:01,352][ INFO][PID:2552227] Downloading results from builder [2025-12-11 13:40:01,353][ INFO][PID:2552227] rsyncing of mockbuilder@34.200.241.165:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc started [2025-12-11 13:40:01,354][ INFO][PID:2552227] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@34.200.241.165:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/ &> /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/build-09899285.rsync.log [2025-12-11 13:40:02,102][ INFO][PID:2552227] rsyncing finished. [2025-12-11 13:40:02,103][ INFO][PID:2552227] Releasing VM back to pool [2025-12-11 13:40:02,113][ INFO][PID:2552227] Searching for 'success' file in resultdir [2025-12-11 13:40:02,113][ ERROR][PID:2552227] Build failed: Backend process error: No success file => build failure [2025-12-11 13:40:02,114][ INFO][PID:2552227] Finished build: id=9899285 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build chroot=centos-stream-10-x86_64 [2025-12-11 13:40:02,116][ ERROR][PID:2552227] Backend process error: No success file => build failure [2025-12-11 13:40:02,116][ INFO][PID:2552227] Worker failed build, took 60.189414501190186 [2025-12-11 13:40:02,117][ INFO][PID:2552227] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "osloup", "project_name": "docbook5_style_xsl_license_and_jar_build", "project_dirname": "docbook5_style_xsl_license_and_jar_build", "submitter": "https://src.fedoraproject.org/user/jakub", "ended_on": 1765460402.1166546, "started_on": 1765460341.9272401, "submitted_on": null, "status": 0, "chroot": "centos-stream-10-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9899285-centos-stream-10-x86_64", "build_id": 9899285, "package_name": "gcc", "package_version": "15.2.1-5", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/osloup/docbook5_style_xsl_license_and_jar_build/gcc", "git_hash": "b4ce49277f8b538f2550e01d3179102adad1765a", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "osloup/docbook5_style_xsl_license_and_jar_build--https://src.fedoraproject.org/user/jakub", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/osloup/docbook5_style_xsl_license_and_jar_build", "result_dir": "09899285-gcc", "built_packages": "", "id": 9899285, "mockchain_macros": { "copr_username": "osloup", "copr_projectname": "docbook5_style_xsl_license_and_jar_build", "vendor": "Fedora Project COPR (osloup/docbook5_style_xsl_license_and_jar_build)" } } ] } [2025-12-11 13:40:02,160][ INFO][PID:2552227] Sending fedora-messaging bus message in build.end [2025-12-11 13:40:02,190][ INFO][PID:2552227] Compressing /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/builder-live.log by gzip [2025-12-11 13:40:02,192][ INFO][PID:2552227] Running command 'gzip /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/builder-live.log' as PID 2594650 [2025-12-11 13:40:02,195][ INFO][PID:2552227] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/builder-live.log) [2025-12-11 13:40:02,196][ INFO][PID:2552227] Compressing /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/backend.log by gzip [2025-12-11 13:40:02,197][ INFO][PID:2552227] Running command 'gzip /var/lib/copr/public_html/results/osloup/docbook5_style_xsl_license_and_jar_build/centos-stream-10-x86_64/09899285-gcc/backend.log' as PID 2594653