[2025-12-16 01:27:01,628][ INFO][PID:122656] Marking build as starting [2025-12-16 01:27:01,917][ INFO][PID:122656] Checking that /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/repodata/repomd.xml exists [2025-12-16 01:27:01,989][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:01,991][ INFO][PID:122656] VM allocation process starts [2025-12-16 01:27:02,018][ INFO][PID:122656] Trying to allocate VM: ResallocHost, ticket_id=359313, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-16 01:27:05,040][ INFO][PID:122656] Allocated host ResallocHost, ticket_id=359313, hostname=18.232.105.216, name=aws_x86_64_spot_prod_00586792_20251216_012531, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-16 01:27:05,054][ INFO][PID:122656] Allocating ssh connection to builder [2025-12-16 01:27:05,055][ INFO][PID:122656] Checking that builder machine is OK [2025-12-16 01:27:05,377][ INFO][PID:122656] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-12-16 01:27:05,599][ INFO][PID:122656] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-12-16 01:27:05,602][ INFO][PID:122656] Filling build.info file with builder info [2025-12-16 01:27:05,605][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:05,606][ INFO][PID:122656] Sending build state back to frontend: { "builds": [ { "timeout": 172800, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rezso", "project_name": "ML", "project_dirname": "ML", "submitter": "rezso", "ended_on": null, "started_on": 1765848425.6067543, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9912831-fedora-rawhide-x86_64", "build_id": 9912831, "package_name": "tvm", "package_version": "0.23.0-20251213.0.git6248b5db.cu13_0", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rezso/ML/tvm", "git_hash": "3e183779fd9de4114b983113a44c35e2926a115e", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rezso/ML--rezso", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rezso/ML/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rezso/CUDA/fedora-rawhide-x86_64/", "id": "copr_rezso_CUDA", "name": "Additional repo copr_rezso_CUDA" }, { "baseurl": "https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64", "id": "https_developer_download_nvidia_com_compute_cuda_repos_rhel9_x86_64", "name": "Additional repo https_developer_download_nvidia_com_compute_cuda_repos_rhel9_x86_64" }, { "baseurl": "https://developer.download.nvidia.com/compute/cuda/repos/rhel9/sbsa", "id": "https_developer_download_nvidia_com_compute_cuda_repos_rhel9_sbsa", "name": "Additional repo https_developer_download_nvidia_com_compute_cuda_repos_rhel9_sbsa" } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rezso/ML", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rezso/ML", "result_dir": "09912831-tvm", "built_packages": "", "id": 9912831, "mockchain_macros": { "copr_username": "rezso", "copr_projectname": "ML", "vendor": "Fedora Project COPR (rezso/ML)" } } ] } [2025-12-16 01:27:05,684][ INFO][PID:122656] Sending fedora-messaging bus message in build.start [2025-12-16 01:27:06,724][ INFO][PID:122656] Sending fedora-messaging bus message in chroot.start [2025-12-16 01:27:06,791][ INFO][PID:122656] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9912831-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-12-16 01:27:07,165][ INFO][PID:122656] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '18.232.105.216' (ED25519) to the list of known hosts. [2025-12-16 01:27:07,168][ INFO][PID:122656] Downloading the builder-live.log file, attempt 1 [2025-12-16 01:27:07,170][ INFO][PID:122656] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.232.105.216 copr-rpmbuild-log [2025-12-16 01:27:12,178][ INFO][PID:122656] Periodic builder liveness probe: alive [2025-12-16 01:27:12,204][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:17,206][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:22,208][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:27,210][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:32,214][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:37,230][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:42,234][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:47,244][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:52,263][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:27:57,270][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:02,271][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:07,272][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:12,273][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:17,276][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:22,280][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:27,286][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:32,321][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:37,325][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:42,343][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:47,346][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:52,350][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:28:57,352][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:02,356][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:07,360][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:12,362][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:17,363][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:22,364][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:27,365][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:32,367][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:37,370][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:42,371][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:47,373][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:52,374][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:29:57,379][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:02,381][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:07,403][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:12,405][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:17,406][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:22,407][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:27,408][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:32,409][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:37,410][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:42,411][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:47,412][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:52,414][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:30:57,414][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:02,415][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:07,417][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:12,417][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:17,418][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:22,420][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:27,421][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:32,422][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:37,423][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:42,424][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:47,425][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:52,426][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:31:57,427][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:02,428][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:07,429][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:12,431][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:17,434][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:22,435][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:27,436][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:32,437][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:37,438][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:42,439][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:47,440][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:52,441][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:32:57,442][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:02,443][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:07,444][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:12,445][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:17,447][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:22,448][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:27,449][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:32,450][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:37,452][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:42,453][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:47,454][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:52,455][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:33:57,456][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:02,457][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:07,458][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:12,459][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:17,460][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:22,460][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:25,984][ ERROR][PID:122656] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-16 01:34:27,461][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:32,463][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:36,007][WARNING][PID:122656] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-16 01:34:37,464][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:42,465][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:46,015][ INFO][PID:122656] Downloading the builder-live.log file, attempt 2 [2025-12-16 01:34:46,116][ INFO][PID:122656] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.232.105.216 copr-rpmbuild-log [2025-12-16 01:34:47,465][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:52,466][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:34:57,467][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:02,468][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:07,469][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:12,470][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:17,471][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:22,472][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:27,473][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:32,474][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:37,475][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:42,476][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:46,157][ ERROR][PID:122656] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-16 01:35:47,477][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:52,478][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:35:56,166][WARNING][PID:122656] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-16 01:35:57,479][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:02,480][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:06,166][ INFO][PID:122656] Downloading the builder-live.log file, attempt 3 [2025-12-16 01:36:06,169][ INFO][PID:122656] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.232.105.216 copr-rpmbuild-log [2025-12-16 01:36:07,480][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:12,481][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:17,482][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:22,483][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:27,484][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:32,485][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:37,486][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:42,487][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:47,489][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:52,490][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:36:57,491][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:02,492][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:06,228][ ERROR][PID:122656] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-16 01:37:07,492][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:12,653][ INFO][PID:122656] Periodic builder liveness probe: alive [2025-12-16 01:37:12,670][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:16,241][WARNING][PID:122656] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-16 01:37:17,671][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:22,671][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:26,242][ INFO][PID:122656] Downloading the builder-live.log file, attempt 4 [2025-12-16 01:37:26,243][ INFO][PID:122656] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.232.105.216 copr-rpmbuild-log [2025-12-16 01:37:27,672][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:32,674][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:37,675][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:42,676][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:47,677][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:52,677][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:37:57,678][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:02,679][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:07,680][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:12,681][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:17,682][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:22,683][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:26,311][ ERROR][PID:122656] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-16 01:38:27,685][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:32,686][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:36,365][ ERROR][PID:122656] Giving up for unstable SSH, failures: 1765848876.002649, 1765848956.1659987, 1765849036.2418942, 1765849116.332661 [2025-12-16 01:38:36,388][ ERROR][PID:122656] Re-try request for task on 'ResallocHost, ticket_id=359313, hostname=18.232.105.216, name=aws_x86_64_spot_prod_00586792_20251216_012531, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-16 01:38:36,391][ INFO][PID:122656] Releasing VM back to pool [2025-12-16 01:38:36,413][ INFO][PID:122656] Retry #1 (on other host) [2025-12-16 01:38:36,431][ INFO][PID:122656] Checking that /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/repodata/repomd.xml exists [2025-12-16 01:38:36,436][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:36,438][ INFO][PID:122656] VM allocation process starts [2025-12-16 01:38:36,475][ INFO][PID:122656] Trying to allocate VM: ResallocHost, ticket_id=359488, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-16 01:38:39,596][ INFO][PID:122656] Allocated host ResallocHost, ticket_id=359488, hostname=52.55.100.253, name=aws_x86_64_spot_prod_00586818_20251216_012648, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-16 01:38:39,599][ INFO][PID:122656] Allocating ssh connection to builder [2025-12-16 01:38:39,601][ INFO][PID:122656] Checking that builder machine is OK [2025-12-16 01:38:39,951][ INFO][PID:122656] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-12-16 01:38:40,158][ INFO][PID:122656] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-12-16 01:38:40,160][ INFO][PID:122656] Filling build.info file with builder info [2025-12-16 01:38:40,169][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:40,177][ INFO][PID:122656] Not re-notifying FE and msg buses for the new host. [2025-12-16 01:38:40,184][ INFO][PID:122656] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9912831-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-12-16 01:38:40,507][ INFO][PID:122656] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '52.55.100.253' (ED25519) to the list of known hosts. [2025-12-16 01:38:40,508][ INFO][PID:122656] Downloading the builder-live.log file, attempt 1 [2025-12-16 01:38:40,509][ INFO][PID:122656] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.55.100.253 copr-rpmbuild-log [2025-12-16 01:38:45,508][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:50,509][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:38:55,510][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:00,511][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:05,514][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:10,515][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:15,516][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:20,517][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:25,519][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:30,520][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:35,521][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:40,522][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:45,523][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:50,524][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:39:55,525][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:00,527][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:05,528][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:10,529][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:15,530][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:20,531][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:25,532][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:30,533][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:35,534][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:40,535][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:45,536][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:50,539][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:40:55,541][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:00,542][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:05,543][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:10,544][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:15,545][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:20,546][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:25,547][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:30,548][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:35,549][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:40,550][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:45,551][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:50,552][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:41:55,554][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:00,634][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:05,635][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:10,636][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:15,637][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:20,638][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:25,639][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:30,640][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:35,641][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:40,642][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:45,644][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:50,646][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:42:55,647][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:00,648][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:05,650][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:10,652][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:15,654][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:20,655][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:25,656][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:30,657][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:35,658][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:40,659][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:45,663][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:50,665][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:43:55,668][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:00,669][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:05,670][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:10,672][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:15,674][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:20,675][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:25,676][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:30,677][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:35,678][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:41,035][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:46,036][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:51,037][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:44:56,038][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:01,039][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:06,040][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:11,041][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:16,042][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:21,043][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:26,044][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:31,045][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:36,046][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:41,073][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:46,074][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:51,075][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:45:56,076][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:01,352][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:06,353][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:11,354][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:16,355][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:21,356][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:26,358][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:31,359][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:36,360][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:41,361][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:46,362][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:51,363][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:46:56,366][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:01,368][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:06,369][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:11,370][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:16,383][ INFO][PID:122656] Periodic builder liveness probe: alive [2025-12-16 01:47:16,384][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:21,385][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:26,386][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:31,387][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:36,388][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:41,389][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:46,390][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:51,391][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:47:56,392][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:01,393][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:06,394][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:11,395][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:16,395][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:21,396][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:26,397][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:31,398][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:36,399][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:41,400][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:46,401][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:51,566][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:48:56,567][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:01,567][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:06,568][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:11,569][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:16,570][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:21,571][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:26,572][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:31,572][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:36,573][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:41,574][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:46,575][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:51,659][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:49:56,660][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:01,661][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:06,662][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:11,663][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:16,664][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:21,665][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:26,666][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:31,667][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:36,668][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:41,669][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:46,670][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:51,769][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:50:56,770][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:01,772][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:06,773][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:11,775][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:16,776][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:21,777][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:26,778][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:31,779][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:36,780][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:41,781][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:46,783][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:51,784][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:51:56,785][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:01,786][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:06,787][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:11,788][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:16,789][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:21,790][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:26,791][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:31,792][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:36,793][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:41,926][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:46,928][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:51,929][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:52:56,930][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:01,931][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:06,932][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:11,933][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:16,934][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:21,937][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:26,956][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:31,958][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:36,959][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:41,960][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:46,961][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:51,962][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:53:56,963][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:01,964][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:06,965][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:11,973][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:16,974][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:21,975][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:26,977][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:31,978][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:36,979][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:42,280][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:47,283][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:52,286][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:54:57,287][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:02,289][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:07,290][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:12,291][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:17,292][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:22,293][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:27,383][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:32,483][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:37,563][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:42,565][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:47,815][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:52,816][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:55:57,817][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:02,818][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:07,818][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:12,819][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:17,820][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:22,821][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:27,822][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:32,823][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:37,824][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:42,825][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:47,826][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:52,827][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:56:57,828][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:02,829][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:07,830][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:12,831][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:17,839][ INFO][PID:122656] Periodic builder liveness probe: alive [2025-12-16 01:57:17,839][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:22,840][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:27,841][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:32,887][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:37,888][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:42,889][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:47,891][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:52,892][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:57:57,894][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:02,895][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:07,896][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:12,898][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:17,900][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:22,902][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:27,903][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:33,255][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:38,257][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:43,258][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:48,259][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:53,260][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:58:58,261][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:03,262][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:08,263][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:13,264][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:18,265][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:23,266][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:28,267][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:33,268][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:38,268][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:43,269][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:48,270][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:53,271][ INFO][PID:122656] Checking for cancel request [2025-12-16 01:59:58,272][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:03,314][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:08,315][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:13,316][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:18,317][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:23,318][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:28,319][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:33,320][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:38,321][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:43,322][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:48,379][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:53,380][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:00:58,381][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:03,382][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:08,384][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:13,384][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:18,395][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:23,396][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:28,397][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:33,451][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:38,455][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:43,466][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:48,467][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:53,468][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:01:58,469][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:03,470][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:08,471][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:13,473][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:18,475][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:23,476][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:28,477][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:33,530][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:38,536][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:43,538][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:48,542][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:53,543][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:02:59,124][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:04,130][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:09,136][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:14,137][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:19,138][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:24,139][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:29,142][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:34,145][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:39,145][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:44,153][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:49,154][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:54,156][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:03:59,158][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:04,160][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:09,163][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:14,164][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:19,179][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:24,183][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:29,184][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:34,197][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:39,206][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:44,208][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:49,220][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:54,233][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:04:59,240][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:04,248][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:09,256][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:14,258][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:19,259][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:24,263][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:29,265][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:34,268][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:39,270][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:44,276][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:49,280][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:54,284][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:05:59,298][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:04,299][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:09,309][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:14,316][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:19,319][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:24,327][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:29,328][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:34,344][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:39,353][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:44,361][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:49,371][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:54,382][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:06:59,396][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:04,403][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:09,411][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:14,424][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:19,460][ INFO][PID:122656] Periodic builder liveness probe: alive [2025-12-16 02:07:19,461][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:24,464][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:29,473][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:34,478][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:39,481][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:44,488][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:49,491][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:54,504][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:07:59,507][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:08:04,515][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:08:09,521][ INFO][PID:122656] Checking for cancel request [2025-12-16 02:08:09,953][ INFO][PID:122656] Downloading results from builder [2025-12-16 02:08:09,966][ INFO][PID:122656] rsyncing of mockbuilder@52.55.100.253:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm started [2025-12-16 02:08:09,973][ INFO][PID:122656] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.55.100.253:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/ &> /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/build-09912831.rsync.log [2025-12-16 02:08:10,643][ INFO][PID:122656] rsyncing finished. [2025-12-16 02:08:10,645][ INFO][PID:122656] Releasing VM back to pool [2025-12-16 02:08:10,755][ INFO][PID:122656] Searching for 'success' file in resultdir [2025-12-16 02:08:10,760][ ERROR][PID:122656] Build failed: Backend process error: No success file => build failure [2025-12-16 02:08:10,866][ INFO][PID:122656] Finished build: id=9912831 failed=True timeout=172800 destdir=/var/lib/copr/public_html/results/rezso/ML chroot=fedora-rawhide-x86_64 [2025-12-16 02:08:10,962][ ERROR][PID:122656] Backend process error: No success file => build failure [2025-12-16 02:08:10,969][ INFO][PID:122656] Worker failed build, took 1770.7957706451416 [2025-12-16 02:08:10,986][ INFO][PID:122656] Sending build state back to frontend: { "builds": [ { "timeout": 172800, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rezso", "project_name": "ML", "project_dirname": "ML", "submitter": "rezso", "ended_on": 1765850890.9678352, "started_on": 1765849120.1720645, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9912831-fedora-rawhide-x86_64", "build_id": 9912831, "package_name": "tvm", "package_version": "0.23.0-20251213.0.git6248b5db.cu13_0", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rezso/ML/tvm", "git_hash": "3e183779fd9de4114b983113a44c35e2926a115e", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rezso/ML--rezso", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rezso/ML/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rezso/CUDA/fedora-rawhide-x86_64/", "id": "copr_rezso_CUDA", "name": "Additional repo copr_rezso_CUDA" }, { "baseurl": "https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64", "id": "https_developer_download_nvidia_com_compute_cuda_repos_rhel9_x86_64", "name": "Additional repo https_developer_download_nvidia_com_compute_cuda_repos_rhel9_x86_64" }, { "baseurl": "https://developer.download.nvidia.com/compute/cuda/repos/rhel9/sbsa", "id": "https_developer_download_nvidia_com_compute_cuda_repos_rhel9_sbsa", "name": "Additional repo https_developer_download_nvidia_com_compute_cuda_repos_rhel9_sbsa" } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rezso/ML", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rezso/ML", "result_dir": "09912831-tvm", "built_packages": "", "id": 9912831, "mockchain_macros": { "copr_username": "rezso", "copr_projectname": "ML", "vendor": "Fedora Project COPR (rezso/ML)" } } ] } [2025-12-16 02:08:11,876][ INFO][PID:122656] Sending fedora-messaging bus message in build.end [2025-12-16 02:08:12,456][ INFO][PID:122656] Compressing /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/builder-live.log by gzip [2025-12-16 02:08:12,477][ INFO][PID:122656] Running command 'gzip /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/builder-live.log' as PID 379164 [2025-12-16 02:08:12,572][ INFO][PID:122656] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/builder-live.log) [2025-12-16 02:08:12,576][ INFO][PID:122656] Compressing /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/backend.log by gzip [2025-12-16 02:08:12,579][ INFO][PID:122656] Running command 'gzip /var/lib/copr/public_html/results/rezso/ML/fedora-rawhide-x86_64/09912831-tvm/backend.log' as PID 379182