Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gTests]Fix GTEST_SKIP kills all gtests in miopen_gtest #3220

Merged
merged 4 commits into from
Sep 4, 2024

Conversation

xinlipn
Copy link
Contributor

@xinlipn xinlipn commented Aug 27, 2024

miopen_gtest wouldn't start on e.g. Navi3x as below. This is because GTEST_SKIP() is invoked in a global environment setup. The fix is to skip the test in TEST_P(). Log with the fix is attached at the bottom.

root@aus-navi3x-09:/MIOpen/build/bin# ./miopen_gtest
Running main() from /opt/rocm/cget/build/tmp-c4f1a42bcee536579a1248d25169f834201170c187fc6d80e8c510a7d28f6f48/googletest-1.14.0/googletest/src/gtest_main .cc
/MIOpen/test/gtest/api_convbiasactiv.cpp:189: Skipped
Skipping fusion test on unsupported ASIC

PRNG seed: 12345678
[==========] Running 8939 tests from 394 test suites.
[----------] Global test environment set-up.
Skipping fusion test on unsupported ASIC

[----------] Global test environment tear-down
[==========] 8939 tests from 394 test suites ran. (0 ms total)
[ PASSED ] 8939 tests.

YOU HAVE 133 DISABLED TESTS

Log with the fix on Navi3x
navi3x.log

@xinlipn xinlipn self-assigned this Aug 27, 2024
@xinlipn xinlipn requested review from JehandadKhan, bghimireamd, BrianHarrisonAMD and CAHEK7 and removed request for JehandadKhan August 27, 2024 06:44
@xinlipn xinlipn added the GTest label Aug 27, 2024
Comment on lines 154 to 156
const auto dev_name = get_handle().GetDeviceName();
#if WORKAROUND_ISSUE_2212
if(!miopen::StartsWith(dev_name, "gfx11") && !miopen::StartsWith(dev_name, "gfx94"))
Copy link
Contributor

@BrianHarrisonAMD BrianHarrisonAMD Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the skip to the start of SetUp(), and use the device flags for skipping instead?

void SetUp() override
{
    if(!IsTestSupportedForDevice())
    {
         GTEST_SKIP();
    }

   // remaining Setup
}



bool IsTestSupportedForDevice()
{
#if WORKAROUND_ISSUE_2212
    using namespace miopen::debug;
    using e_mask = enabled<Gpu::gfx900, Gpu::gfx906, Gpu::gfx908, Gpu::gfx90A, Gpu::gfx103X>;
    using d_mask = disabled<Gpu::gfx110X, Gpu::gfx94X>;
   return ::IsTestSupportedForDevMask<d_mask, e_mask>();
#else
    return true;
#endif
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a part of overall gtest improvements https://github.com/ROCm/MIOpen/wiki/GTest-development#early-skip
Probably we should add some tickets here #3140

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the skip to the start of SetUp(), and use the device flags for skipping instead?

void SetUp() override
{
    if(!IsTestSupportedForDevice())
    {
         GTEST_SKIP();
    }

   // remaining Setup
}



bool IsTestSupportedForDevice()
{
#if WORKAROUND_ISSUE_2212
    using namespace miopen::debug;
    using e_mask = enabled<Gpu::gfx900, Gpu::gfx906, Gpu::gfx908, Gpu::gfx90A, Gpu::gfx103X>;
    using d_mask = disabled<Gpu::gfx110X, Gpu::gfx94X>;
   return ::IsTestSupportedForDevMask<d_mask, e_mask>();
#else
    return true;
#endif
}

@BrianHarrisonAMD , thanks for the comments. Code updated

Comment on lines 188 to 196
void GatherCBATestCases(std::vector<CBATestCase>& cba_test_cases)
{
cba_test_cases.push_back(CBATestCase{
16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution});
}

// Extra layer of indirection introduced since GTEST_SKIP() cannot be called from non-void function.
std::vector<CBATestCase> GetTestValues()
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could combine these into a single method that returns the testcases, and I think the comment isn't applicable anymore with this fix.

std::vector<CBATestCase> GetTestValues()
{
    return {{ 16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution }};
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could combine these into a single method that returns the testcases, and I think the comment isn't applicable anymore with this fix.

std::vector<CBATestCase> GetTestValues()
{
    return {{ 16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution }};
}

@BrianHarrisonAMD , thanks for the comments. Cleaned up code

@BrianHarrisonAMD
Copy link
Contributor

Couple minor comments, but looks good!

Interesting find!
I had no idea it worked that way hehe.

@xinlipn xinlipn changed the title [Test]Fix GTEST_SKIP kills all gtests in miopen_gtest [gTests]Fix GTEST_SKIP kills all gtests in miopen_gtest Aug 27, 2024
Copy link
Contributor

@atamazov atamazov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 LGTM

@junliume junliume merged commit e5109c2 into develop Sep 4, 2024
143 checks passed
@junliume junliume deleted the sl/gtest_skipped branch September 4, 2024 04:50
randyspauldingamd pushed a commit that referenced this pull request Feb 18, 2025
* [Windows] Debug vector add Windows CI failure. (#3090)

* fixed missing <CR> in the kernel

* replaced uint with size_t to fix Windows CI

* clang format fix

* Fixed duplicating info, narrowing conversion.

* [MHA] Implement MIOPEN_BACKEND_OPERATION_RESHAPE_DESCRIPTOR (#3082)

* Implement MIOPEN_BACKEND_OPERATION_RESHAPE_DESCRIPTOR

* Introduce an enum to indicate transpose op

* Fix possible throws in the test

* Bump rocm-docs-core[api_reference] from 1.4.1 to 1.5.0 in /docs/sphinx (#3101)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.4.1...v1.5.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump certifi from 2024.2.2 to 2024.7.4 in /docs/sphinx (#3102)

Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.2.2 to 2024.7.4.
- [Commits](certifi/python-certifi@2024.02.02...2024.07.04)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix #3076 - issues related to in_offset overflow in Col2Im (2D and 3D) and Im2Col (2D only) (#3099)

* fix-overflow-col2im2d(01) Switch to unsigned math in 2D and 3D col2im kernels.

* fix-overflow-col2im2d(02) Add "dry-run" mode to Col2ImGPU() and use it.

* fix-overflow-col2im2d(03) Revert "fix-overflow-col2im2d(02) Add "dry-run" mode to Col2ImGPU() and use it."

This reverts commit c8f9d40.

* fix-overflow-col2im2d(04) Add 64-bit index mode

* fix-overflow-col2im2d(06) Tidy fix

* fix-overflow-col2im2d(07) Minimal changes to avoid trimming of im_offset in 2D Im2Col

* Update CK commit hash in requirements.txt (#3087)

* Update CK commit hash in requirements.txt

* update CK commit to include fix

* use CK version with cmake fixes

* update CK commit hash to include the fix

---------

Co-authored-by: illsilin <[email protected]>

* Enable hipBLASLt backend for GEMM, and make it the default option for RNN's under specific conditions (#3030)

* Unblock CI by regenerating gfx908.kdb (#3092)

* Support for invoker caching and find modes to fusion find (#3095)

* implemented support for invoker caching and find modes to fusion find

* Fixed some errors

* [NFC] Move convolution solvers to solver/conv directory (part 2) (#3042)

* [Fixes CI] Fixed tidy hang (#3114)

* Fixed tidy hang by disabling a check on bugged versions

* Added a comment on why this is necessary

* Find: Optimize benchmarking for long execution times. (#3103)

* [ROCm 6.2] add change log (#3113)

* [WA] Remove exception checking tests for unsupported hardware (#3117)

* Remove exception checking tests for unsupported hardware

* Add workaround define and re-add tests

* Add tests for ConvBiasResAddActivation forward using Graph API (#3112)

* Add tests for ConvBiasResAddActivation forward using Graph API

* Apply formatting

* [gfx12] add support of gfx12 platforms (#3109)

* [Staging] Update CK commit hash in requirements.txt (#3122)

* Revert "[Staging] Update CK commit hash in requirements.txt (#3122)" (#3125)

This reverts commit 8449363.

* Update GraphAPI Tensor to inherit from TensorDescriptor (#3119)

* [COMgr] Prepare for COMgr 3.x (#3107)

* amdcomgr-3(01) [cmake] Do not allow different values of MIOPEN_USE_COMGR and MIOPEN_USE_HIPRTC. Remove leftover of support of ROCm older than 5.0.

* amdcomgr-3(02) [importance_normal] Fix precompiled binary cache miss for Winograd Fury. Resolves #2778 (comment)

* amdcomgr-3(03) [winograd fury] Add comment.

* amdcomgr-3(04) [comgr] Removed comgr::BuildHip()

* amdcomgr-3(05) [comgr] Added support for AMD COMgr 3.0. Removed support for AMD COMgr older than 1.7.

* amdcomgr-3(06) [comgr] Removed support of unused enum members from to_string().

* amdcomgr-3(08) [comgr] Fix tidy error

---------

Co-authored-by: Evgenii Averin <[email protected]>

* [debugging] Make naive conv solvers obey MIOPEN_DEBUG_HIP_KERNELS (#3111)

* Fix GCC8 linking issues with stdc++fs (#3126)

* Added missing weight sorting as WTI calculation in fusion (#3105)

* GTests renaming: filename starting letters A-B (#3124)

* Tests with starting letters of the filenames "a"-"b" are renamed

* "b" test change added

---------

Co-authored-by: Jun Liu <[email protected]>

* [bug][tests] Fix broken tests (#3123)

* Fix broken gtests

* Fix formatting

* Fix ctests

---------

Co-authored-by: Jun Liu <[email protected]>

* Changed Fusion to have a separate find mode env variable and default to fast (#3121)

* Added logging of paths to HIP, HIPRTC, COMgr, rocBlas, rocMLIR, rocRand, frugally-deep, Eigen3. Added logging of versions of frugally-deep, Eigen3. (#3110)

* Implement getitem backward (#2883)

* Fix bnorm args log info (#3127)

Co-authored-by: Alex Eremin <[email protected]>

* Fixed redundant find on failed fallback (#3131)

Co-authored-by: Jun Liu <[email protected]>

* [gtest] Fix log test error (#3150)

* Bump rocm-docs-core[api_reference] from 1.5.0 to 1.6.1 in /docs/sphinx (#3160)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.5.0 to 1.6.1.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.5.0...v1.6.1)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Windows] Temporary workaround on rocMLIR issue (#3164)

* [RNN] BWD concurrent imp (#3169)

* GTests renaming: filenames which start with letter C (#3153)

* [MHA] Implement MHA backward pattern (#3128)

* Reduce calculation prod enhancement in case of inner dim (#2970)

* Deduplicated ValidateGroupCount and fixed incorrect usage (#3177)

* Deduplicated ValidateGroupCount

* Changed static to namespace {}

* format

* Fixed find 2.0 ValidateGroupCount incorrect usage when transposed

* windows fix, build without GEMM (#3179)

* windows fix

* Copyright

* Refactor BnCKFwdInference::GetSolution for NHWC (#3120)

* [WORKAROUND] Allow to use boost:filesystem instead of std::filesystem, [BUGFIX] check_cxx_linker_flag (#3154)

* GTests renaming: filenames which start with letters D-G (#3178)

* RNN wall clock timer update (#3180)

* Add pattern and executor for ConvBiasResAddActiv GraphAPI (#3161)

* [RNN] WA for tensors with more than 3 dims. (#3186)

* [Fix][TransformTensor] Ignore output buffer when BETA=0 (#3184)

* MI300 TunaNet Update: CK FWD and WRW Solvers Updated (#3130)

* Bump rocm-docs-core[api_reference] from 1.6.1 to 1.6.2 in /docs/sphinx (#3185)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.6.1 to 1.6.2.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/v1.6.2/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.6.1...v1.6.2)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Dropout kernel OpenCL to HIP + gtest (#3088)

* Enhancement rotary position embedding (#3009)

* [RNN] MS heuristic update (#3187)

* Update Conv3DTestCase to use structs (#3162)

* Backward MHA test using C++ API (#3097)

* Revert "[Fix][TransformTensor] Ignore output buffer when BETA=0 (#3184)" (#3194)

This reverts commit a89a850.

* Update CK without new base ROCm renew (#3191)

* [gtest] Unit test for ConvWinoFuryRxS (#3175)

* Update CK commit hash to include fix for gfx12 (#3195)

* [Perf] replace blocking hipMemset with async hip Memset (#3196)

* Bump rocm-docs-core[api_reference] from 1.6.2 to 1.7.0 in /docs/sphinx (#3201)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.6.2 to 1.7.0.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.6.2...v1.7.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Enable Graph execution in Backward MHA C++ API Test (#3199)

* WIP: add reshape nodes and enalbe graph execution

* fixed a bug due to polluted variant pack

* rename for consistency

* [WA] Disable MLIR when building debug to workaround sanitizer issue (#3200)

* Update docker ROCm version for CI and MIOpen version (#3181)

* Implement PReLU backward (#3152)

* Bump rocm-docs-core[api_reference] from 1.7.0 to 1.7.1 in /docs/sphinx (#3205)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.7.0...v1.7.1)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [gtest] Unit tests for GEMM solvers (#3197)

* Fix bn layout network config caching issue (#3203)

* Reapplied and fixed #3194 (#3211)

* Reapply "[Fix][TransformTensor] Ignore output buffer when BETA=0 (#3184)" (#3194)

This reverts commit 326d734.

* Fixed incorrect network config

* Removed redundant boost::thread include (#3210)

* Fixed Find 2.0 fusion ignoring miopenSetFindOptionTuning (#3157)

* [Tests] All tests in the same test suite must use the same test fixture class (#3202)

class

* GTests renaming: filenames which start with letters H-M (#3209)

* GTests renaming: filenames which start with letters N-R (#3216)

* Add gtest name checker (#3218)

* Gtest header patch for pre-c++17 compatibility (#3223)

* patch gtest header for compatibility with pre-c++17

* use fs in place of filesystem

* namespace for fs

* GTests renaming: filenames which start with letter S (#3221)

* Bump rocm-docs-core[api_reference] from 1.7.1 to 1.7.2 in /docs/sphinx (#3214)

Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](ROCm/rocm-docs-core@v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* GenericSearch Patience (#3215)

* [Tests] skip cba test for unsupported platforms (#3224)

* Fix implementation with average and update reference links (#3231)

* [gTests]Fix GTEST_SKIP kills all gtests in miopen_gtest (#3220)

* Fix GTEST_SKIP kills all gtests in miopen_gtest

* Moved GTEST_SKIP in Setup() and removed unneeded code

* [RNN] LSTM backward weight update (#3219)

* GTests renaming: Letters "T"-"Z". Enabling name checker (#3229)

* T-Z renamed

* some tests are renamed

* python script fix

* KTN accounts for CK Fwd type additn (#3206)

Co-authored-by: Chris Erb <[email protected]>

* Initial porting of Op2dTensorGeneric kernel from OpenCL to HIP (#3222)

* initial rewrite of Op2dTensorGeneric to HIP

* Changes made according to suggestions

* template and network_config update

* Improvements to the test, added performance testing

* Additional test improvements

* fixed formatting issue

* Change to support build with offline compiler

* Change from hardcoded value to max_num_wg variable

* Format issue fixed

---------

Co-authored-by: Alex Eremin <[email protected]>

* Bump cryptography from 42.0.7 to 43.0.1 in /docs/sphinx (#3233)

Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.7 to 43.0.1.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](pyca/cryptography@42.0.7...43.0.1)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* TensorDescriptor refactoring & enhancements (#3213)

* Fix ConvBiasResAddActivation_fwd tests failing on Navi (#3239)

* Fix ConvBiasResAddActivation_fwd tests failing on Navi

* Update to use IsTestSupportedByDevice instead

* Update test to have it's own supported device check

* Fix clang-format-12 issue

---------

Co-authored-by: Jun Liu <[email protected]>

* Fix stream not being set when calling hipMemsetAsync (#3243)

* Fix stream not being set when calling hipMemsetAsync

* fix clang-format issue

---------

Co-authored-by: Jun Liu <[email protected]>

* Fix gfx908 timing out during pipeline stages (#3245)

* Fix gfx908 timing out during pipeline stages

* Limit timeout increase

* [RNN] LSTM backward weight MS (#3241)

* Update CK branch to include CK Fwd changes (#3236)

* Mha Bias added to Find 2.0 for forward pass (#3240)

* [BugFix] Get kernel after the stream change (#3248)

* Dropout replacement finish (#3228)

* Update CK commit to include legacy OS fix (#3258)

* TunaNet update (#3252)

* [MHA] add bias (#3251)

* [Bug] missing split_k from end of kernel_id in ck wrw (#3249)

* append split_k to kernel_id for ck wrw

* fix clang format issue

---------

Co-authored-by: Jun Liu <[email protected]>

* merge corrections

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: sgundabo <[email protected]>
Co-authored-by: Sergei Grigoriev <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Artem Tamazov <[email protected]>
Co-authored-by: Jun Liu <[email protected]>
Co-authored-by: illsilin <[email protected]>
Co-authored-by: BrianHarrisonAMD <[email protected]>
Co-authored-by: Vasilii Filippov <[email protected]>
Co-authored-by: Evgenii Averin <[email protected]>
Co-authored-by: mentat <[email protected]>
Co-authored-by: Vsevolod Golovko <[email protected]>
Co-authored-by: Seungman Han <[email protected]>
Co-authored-by: who who who <[email protected]>
Co-authored-by: Alex Eremin <[email protected]>
Co-authored-by: Artur Wojcik <[email protected]>
Co-authored-by: Kamil Nasyrov <[email protected]>
Co-authored-by: Vasilii Filippov <[email protected]>
Co-authored-by: xinlipn <[email protected]>
Co-authored-by: M. Saud Ul Hassan <[email protected]>
Co-authored-by: amberhassaan <[email protected]>
Co-authored-by: Daming Feng <[email protected]>
Co-authored-by: Long Luong <[email protected]>
Co-authored-by: Djordje Novakovic <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants