LLM Harness Evaluation #526
zhentaoccllm-harness-evaluation.yml
on: schedule
llm-cpp-build
/
check-linux-amx-artifact
3s
llm-cpp-build
/
check-linux-avx512-artifact
7s
llm-cpp-build
/
check-linux-avxvnni-artifact
2s
llm-cpp-build
/
check-windows-avx-artifact
0s
llm-cpp-build
/
check-windows-avx-vnni-artifact
0s
llm-cpp-build
/
check-windows-avx2-artifact
0s
set-matrix
0s
Matrix: llm-harness-evaluation
Annotations
4 errors and 51 warnings
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, arc, sym_int4, xpu)
unable to access 'https://github.com/intel-analytics/ipex-llm/': server certificate verification failed. CAfile: none CRLfile: none
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, arc, sym_int4, xpu)
Process completed with exit code 1.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, arc, fp8, xpu)
Process completed with exit code 128.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, arc, sym_int4, xpu)
Process completed with exit code 1.
|
|
llm-cpp-build / linux-build-amx
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-cpp-build / linux-build-avxvnni
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-cpp-build / linux-build-avx512
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, sym_int4, xpu)
Failed to download action 'https://api.github.com/repos/actions/checkout/tarball/f43a0e5ff2bd294095638e18286ca9a3d1956744'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, sym_int4, xpu)
Back off 12.498 seconds before retry.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, truthfulqa, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, truthfulqa, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, truthfulqa, fp8, xpu)
Failed to download action 'https://api.github.com/repos/actions/download-artifact/tarball/9bc31d5ccc31df68ecc42ccf4149144866c47d8a'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, truthfulqa, fp8, xpu)
Back off 23.083 seconds before retry.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, fp8, xpu)
Failed to download action 'https://api.github.com/repos/actions/upload-artifact/tarball/a8a3f3ad30e3422c9c7b888a15615d19a852ae32'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, winogrande, fp8, xpu)
Back off 27.766 seconds before retry.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, arc, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Baichuan2-7B-Chat-LLaMAfied, arc, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, truthfulqa, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, truthfulqa, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, winogrande, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, winogrande, sym_int4, xpu)
Failed to download action 'https://api.github.com/repos/actions/checkout/tarball/f43a0e5ff2bd294095638e18286ca9a3d1956744'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, winogrande, sym_int4, xpu)
Back off 14.562 seconds before retry.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, arc, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, arc, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, falcon-7b-instruct-with-patch, winogrande, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, truthfulqa, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, arc, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, truthfulqa, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, winogrande, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, winogrande, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Llama2-7b-guanaco-dolphin-500, arc, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, arc, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, truthfulqa, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, arc, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, arc, fp8, xpu)
Failed to download action 'https://api.github.com/repos/actions/upload-artifact/tarball/a8a3f3ad30e3422c9c7b888a15615d19a852ae32'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, arc, fp8, xpu)
Back off 22.344 seconds before retry.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, truthfulqa, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, fp8, xpu)
Failed to download action 'https://api.github.com/repos/actions/checkout/tarball/f43a0e5ff2bd294095638e18286ca9a3d1956744'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, fp8, xpu)
Back off 14.396 seconds before retry.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, arc, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, arc, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, sym_int4, xpu)
Failed to download action 'https://api.github.com/repos/actions/download-artifact/tarball/9bc31d5ccc31df68ecc42ccf4149144866c47d8a'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, Mistral-7B-v0.1, winogrande, sym_int4, xpu)
Back off 25.076 seconds before retry.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, truthfulqa, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, truthfulqa, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, truthfulqa, sym_int4, xpu)
Failed to download action 'https://api.github.com/repos/actions/upload-artifact/tarball/a8a3f3ad30e3422c9c7b888a15615d19a852ae32'. Error: The SSL connection could not be established, see inner exception.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, truthfulqa, sym_int4, xpu)
Back off 18.727 seconds before retry.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, winogrande, fp8, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-evaluation (3.11, mpt-7b-chat, winogrande, sym_int4, xpu)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
llm-harness-summary
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744, actions/setup-python@v4, actions/download-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
|
|
Deprecation notice: v1, v2, and v3 of the artifact actions
The following artifacts were uploaded using a version of actions/upload-artifact that is scheduled for deprecation: "harness_results", "linux-amx", "linux-avx", "linux-avx2", "linux-avx512", "linux-avxvnni".
Please update your workflow to use v4 of the artifact actions.
Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
harness_results
Expired
|
14.4 KB |
|
|
linux-amx
Expired
|
4.3 MB |
|
|
linux-avx
Expired
|
2.25 MB |
|
|
linux-avx2
Expired
|
2.19 MB |
|
|
linux-avx512
Expired
|
4.23 MB |
|
|
linux-avxvnni
Expired
|
4.93 MB |
|