You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I need to use sentencepiece for tokenization, and I also need OpenVINO for NLP task inference. I am using vcpkg to manage both sentencepiece and OpenVINO. The protobuf for OpenVINO comes from vcpkg, while the protobuf for sentencepiece defaults to internal rather than package. When I link both OpenVINO and sentencepiece libraries, I encounter symbol redefinition issues. Therefore, I have to change the SPM_PROTOBUF_PROVIDER of sentencepiece to "package" to use protobuf from vcpkg. However, after making this change, the compilation fails. How can I resolve this issue?
cmake change
@@ -33,7 +33,7 @@ option(SPM_NO_THREADLOCAL "Disable thread_local operator"OFF)
option(SPM_ENABLE_MSVC_MT_BUILD, "Use /MT flag in MSVC build"OFF)
option(SPM_CROSS_SYSTEM_PROCESSOR, "Override system processor""")
-set(SPM_PROTOBUF_PROVIDER "internal" CACHE STRING "Provider of protobuf library")
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING "Provider of protobuf library")
set_property(CACHE SPM_PROTOBUF_PROVIDER PROPERTYSTRINGS"internal""package")
set(SPM_ABSL_PROVIDER "internal"CACHE STRING "Provider of absl library")
set_property(CACHE SPM_ABSL_PROVIDER PROPERTYSTRINGS"internal""module""package")
log
···
[build] C:\PROGRA1\MICROS1\2022\COMMUN1\VC\Tools\MSVC\14401.338\bin\Hostx86\x64\cl.exe /nologo /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\cpp_faster\sentencepiece -IH:\cpp_faster\sentencepiece\out\build\win-x64-release -IH:\cpp_faster\sentencepiece\third_party\absl -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\cpp_faster\sentencepiece\out\build\win-x64-release\src -IH:\cpp_faster\sentencepiece\src..\third_party /DWIN32 /D_WINDOWS /W3 /GR /EHsc /O2 /Ob2 /DNDEBUG -std
++17 -MD /wd4267 /wd4244 /wd4305 /Zc
/utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece-static.dir\error.cc.obj /Fdsrc\CMakeFiles\sentencepiece-static.dir\sentencepiece-static.pdb /FS -c H:\cpp_faster\sentencepiece\src\error.cc
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2737: “absl::lts_20240116::type_identity_t”: constexpr object must be initialized
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “{”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2447: “{”: missing functionheader (old-style formal list?)
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “)” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2059: syntax error: “)”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(755): error C2091: functionreturns function
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(761): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
···
C:\PROGRA~1\MICROS~1\2022\COMMUN~1\VC\Tools\MSVC\1440~1.338\bin\Hostx64\x64\cl.exe /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg\src -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\..\third_party /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP /D_DEBUG /MTd /Z7 /Ob0 /Od /RTC1 -std:c++17 -MTd /wd4267 /wd4244 /wd4305 /Zc:strictStrings /utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece_train-static.dir\unigram_model_trainer.cc.obj /Fdsrc\CMakeFiles\sentencepiece_train-static.dir\sentencepiece_train-static.pdb /FS -c H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\unigram_model_trainer.cc
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(58): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or 'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(244): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or 'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(248): error C2872: 'string_view': ambiguous symbol
···
The text was updated successfully, but these errors were encountered:
I need to use sentencepiece for tokenization, and I also need OpenVINO for NLP task inference. I am using vcpkg to manage both sentencepiece and OpenVINO. The protobuf for OpenVINO comes from vcpkg, while the protobuf for sentencepiece defaults to internal rather than package. When I link both OpenVINO and sentencepiece libraries, I encounter symbol redefinition issues. Therefore, I have to change the SPM_PROTOBUF_PROVIDER of sentencepiece to "package" to use protobuf from vcpkg. However, after making this change, the compilation fails. How can I resolve this issue?
cmake change
log
The text was updated successfully, but these errors were encountered: