Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use system deps in linux CI #2068

Closed
wants to merge 1 commit into from
Closed

use system deps in linux CI #2068

wants to merge 1 commit into from

Conversation

ahornby
Copy link
Contributor

@ahornby ahornby commented Sep 18, 2023

use system deps in linux CI

Speed up the CI by not rebuilding boost et al each time.

Test plan:

regenerate github actions with:

./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --os-type=linux --output-dir=.github/workflows folly

build locally with system dependencies

./build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive folly
./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=.  folly

After, its quicker as we can use system boost


Stack created with Sapling. Best reviewed with ReviewStack.

@ahornby ahornby changed the title update googletest manifest so folly tests build update googletest manifest so folly tests build on linux Sep 19, 2023
@ahornby ahornby changed the title update googletest manifest so folly tests build on linux fix build failure on ubuntu with system packages Sep 19, 2023
@ahornby ahornby force-pushed the pr2068 branch 2 times, most recently from 1e61557 to 7035f3b Compare September 23, 2023 11:34
This was referenced Sep 23, 2023
@ahornby ahornby force-pushed the pr2068 branch 3 times, most recently from f065820 to 8341303 Compare October 1, 2023 19:19
@facebook-github-bot
Copy link
Contributor

@Orvid has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@ahornby ahornby changed the title fix build failure on ubuntu with system packages use system deps in linux CI Oct 13, 2023
@ahornby ahornby mentioned this pull request Oct 16, 2023
@ahornby ahornby force-pushed the pr2068 branch 2 times, most recently from 8b81eec to f871235 Compare October 20, 2023 18:23
facebook-github-bot pushed a commit that referenced this pull request Oct 23, 2023
Summary:
free up disk space to allow linux CI to pass

Free up some disk from the runner and intermediate build steps as folly linux CI link step is failing with out of disk space.

Regenerated CI yaml with:  ./build/fbcode_builder/getdeps.py generate-github-actions --src-dir=. --free-up-disk --os-type=linux --output-dir=.github/workflows folly

 ---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/folly/pull/2070).
* #2084
* #2074
* #2068
* #2073
* __->__ #2070

Pull Request resolved: #2070

Reviewed By: genevievehelsel

Differential Revision: D50568487

Pulled By: xavierd

fbshipit-source-id: 271478cc5f279f220acbedc2a7d3f1252effd4b9
facebook-github-bot pushed a commit to facebook/hhvm that referenced this pull request Oct 23, 2023
Summary:
free up disk space to allow linux CI to pass

Free up some disk from the runner and intermediate build steps as folly linux CI link step is failing with out of disk space.

Regenerated CI yaml with:  ./build/fbcode_builder/getdeps.py generate-github-actions --src-dir=. --free-up-disk --os-type=linux --output-dir=.github/workflows folly

 ---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/folly/pull/2070).
* facebook/folly#2084
* facebook/folly#2074
* facebook/folly#2068
* facebook/folly#2073
* __->__ facebook/folly#2070

X-link: facebook/folly#2070

Reviewed By: genevievehelsel

Differential Revision: D50568487

Pulled By: xavierd

fbshipit-source-id: 271478cc5f279f220acbedc2a7d3f1252effd4b9
@ahornby ahornby force-pushed the pr2068 branch 3 times, most recently from 7152ce8 to bf93582 Compare December 3, 2023 21:09
@facebook-github-bot
Copy link
Contributor

@Orvid has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot pushed a commit that referenced this pull request Dec 13, 2023
Summary:
fix linux and macOS build

linux and mac build was failing:
   * gcc and xcode hide std::chrono::years if C++ 20 mode not set.  Disable the affected test in this case
   * same with std::latch, also disable the affected test in this case

macOS build was failing with:

   * missing header includes, added them

macOS test run was failing with:
   * libcpp exception text has changed from "with" to "due to" which was breaking ExceptionTest.cpp, update the expectation to cover both.
   * FileUtilDetail.cpp was only enabling custom temp dir on linux, breaking FileUtilTest.cpp.  Enable custom temp dir for non-windows (i.e. mac and other unix) to fix it.

Pull Request resolved: #2073

Test Plan:
CI.  Before, build error on vector
```
CMakeFiles/tdigest_benchmark.dir/folly/stats/test/TDigestBenchmark.cpp.o -MF CMakeFiles/tdigest_benchmark.dir/folly/stats/test/TDigestBenchmark.cpp.o.d -o CMakeFiles/tdigest_benchmark.dir/folly/stats/test/TDigestBenchmark.cpp.o -c /Users/runner/work/folly/folly/folly/stats/test/TDigestBenchmark.cpp
In file included from /Users/runner/work/folly/folly/folly/stats/test/TDigestBenchmark.cpp:17:
/Users/runner/work/folly/folly/folly/stats/DigestBuilder.h:59:25: error: implicit instantiation of undefined template 'std::vector<double>'
    std::vector<double> buffer;
```

then build error on array
```
CMakeFiles/constexpr_math_test.dir/folly/test/ConstexprMathTest.cpp.o -c /Users/runner/work/folly/folly/folly/test/ConstexprMathTest.cpp
/Users/runner/work/folly/folly/folly/test/ConstexprMathTest.cpp:38:29: error: implicit instantiation of undefined template 'std::array<unsigned long, 7>'
    std::array<res_t, size> res{};
```

the build error on std::chrono::year
```
/Users/alex/local/folly.sapling/folly/logging/test/RateLimiterTest.cpp:124:47: error: no member named 'years' in namespace 'std::chrono'
  IntervalRateLimiter limiter{1, std::chrono::years{1}};
```

then test failues

After, works

 ---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/folly/pull/2073).
* #2074
* #2068
* #2084
* __->__ #2073

Reviewed By: xavierd

Differential Revision: D49931137

Pulled By: Orvid

fbshipit-source-id: 58830a637e73375eb648c5897dd8a1e5e479c657
facebook-github-bot pushed a commit that referenced this pull request Feb 16, 2024
Summary:
fix linux cmake test disablement and related OOM

Fix the folly linux github CI:

   * fix disabling the failing AtomicUnorderedInsertMap.DISABLEDMegaMap test as ctest wasn't filtering it out
   * fix disabling ThreadLocal.SHAREDLibraryTestName when not building shared lib

The first one also fixes OOM that reliably OOM killed my terminal when running tests locally with > 2 jobs

Pull Request resolved: #2146

Test Plan:
Run tests locally with:
```
./build/fbcode_builder/getdeps.py --allow-system-packages test --num-jobs 2 --src-dir=. folly
```

Before:
```
Start 1883: atomic_unordered_map_test.AtomicUnorderedInsertMap.DISABLEDMegaMap
1882/3090 Test #1883: atomic_unordered_map_test.AtomicUnorderedInsertMap.DISABLEDMegaMap ...................................***Failed    0.00 sec
Note: Google Test filter = AtomicUnorderedInsertMap.DISABLEDMegaMap
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from AtomicUnorderedInsertMap
[ RUN      ] AtomicUnorderedInsertMap.DISABLEDMegaMap
unknown file: Failure
C++ exception with description "Cannot allocate memory" thrown in the test body.
[  FAILED  ] AtomicUnorderedInsertMap.DISABLEDMegaMap (0 ms)
[----------] 1 test from AtomicUnorderedInsertMap (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] AtomicUnorderedInsertMap.DISABLEDMegaMap

 1 FAILED TEST
...
1/1 Test #2774: thread_local_test.ThreadLocal.SHAREDLibraryTestName ...***Failed    0.00 sec
Note: Google Test filter = ThreadLocal.SHAREDLibraryTestName
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from ThreadLocal
[ RUN      ] ThreadLocal.SHAREDLibraryTestName
unknown file: Failure
C++ exception with description "missing: folly/test/thread_local_test_lib.so" thrown in the test body.
[  FAILED  ] ThreadLocal.SHAREDLibraryTestName (0 ms)
[----------] 1 test from ThreadLocal (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] ThreadLocal.SHAREDLibraryTestName

```

After, works

 ---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/folly/pull/2146).
* #2084
* #2074
* #2068
* __->__ #2146

Reviewed By: Gownta

Differential Revision: D53830611

Pulled By: Orvid

fbshipit-source-id: c3eac882f2522996f897b29f5bf7d8622c1f6ceb
Speed up the CI by not rebuilding boost et al each time.

Test plan:

regenerate github actions with:
```
./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --os-type=linux --output-dir=.github/workflows folly
```

build locally with system dependencies
```
./build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive folly
./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=.  folly
```

After, its quicker as we can use system boost
@facebook-github-bot
Copy link
Contributor

@Orvid has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@Orvid merged this pull request in 5ee12d7.

facebook-github-bot pushed a commit to facebook/hhvm that referenced this pull request Feb 21, 2024
Summary:
use system deps in linux CI

Speed up the CI by not rebuilding boost et al each time.

X-link: facebook/folly#2068

Reviewed By: Gownta

Differential Revision: D49931071

Pulled By: Orvid

fbshipit-source-id: cfdc91ca7da94abe5a3335b0b2826318d68ce52c
facebook-github-bot pushed a commit that referenced this pull request Feb 21, 2024
Summary:
use system deps in mac CI

lets save the boost build time during CI on mac as well

regenerated the actions with: ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --src-dir=. --free-up-disk --os-type=darwin --output-dir=.github/workflows folly

Pull Request resolved: #2074

Test Plan:
CI

Before:  build works, some intermittent failing tests
After: same but we get the CI result quicker

 ---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/folly/pull/2074).
* __->__ #2074
* #2068

Reviewed By: Gownta

Differential Revision: D52090738

Pulled By: Orvid

fbshipit-source-id: 78a32dd4a25f43f0d99ea6b80c4dc0fdca65e511
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants