[libclang] Always Dup in createRef(StringRef)

[vitalybuka]

We can't guaranty that underlying string is
0-terminated and [String.size()] is even in the
same allocation.

https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio

==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0  in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36
    #1  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39
    #2  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7
    #3  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7
    #4  in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3
    #5  in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7

  Memory was marked as uninitialized
    #0  in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5
    #1  in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7
    #2  in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12
    #3  in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43
    #4  in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29
    #5  in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30
    #6  in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10
    #7  in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16
    #8  in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9
    #9  in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp
    #10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22
    #11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12
    #12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35
    #13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35
    #14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25

[llvmbot]

@llvm/pr-subscribers-clang

Author: Vitaly Buka (vitalybuka)

Changes

We can't guaranty that underlying string is
0-terminated and [String.size()] is even in the
same allocation.

---

Full diff: https://github.com/llvm/llvm-project/pull/125020.diff

1 Files Affected:

• (modified) clang/tools/libclang/CXString.cpp (+1-13)

diff --git a/clang/tools/libclang/CXString.cpp b/clang/tools/libclang/CXString.cpp
index 5e427957a1092b..aaa8f8eeb67a12 100644
--- a/clang/tools/libclang/CXString.cpp
+++ b/clang/tools/libclang/CXString.cpp
@@ -87,19 +87,7 @@ CXString createRef(StringRef String) {
   if (String.empty())
     return createEmpty();
 
-  // If the string is not nul-terminated, we have to make a copy.
-
-  // FIXME: This is doing a one past end read, and should be removed! For memory
-  // we don't manage, the API string can become unterminated at any time outside
-  // our control.
-
-  if (String.data()[String.size()] != 0)
-    return createDup(String);
-
-  CXString Result;
-  Result.data = String.data();
-  Result.private_flags = (unsigned) CXS_Unmanaged;
-  return Result;
+  return createDup(String);
 }
 
 CXString createDup(StringRef String) {

[vitalybuka]

It does not look nice for memory usage, but I don't know how critical it's here.
It's it's unacceptable, then __attribute__((no_sanitize("memory"))) is an option.