Opened 8 years ago

Closed 5 years ago

#713 closed defect (fixed)

valgrind leak checking false positives on OS X 10.11

Reported by: German M. Bravo Owned by: Olly Betts
Priority: normal Milestone: 1.4.12
Component: Test Suite Version: git master
Severity: normal Keywords:
Cc: James Aylett Blocked By:
Blocking: Operating System: Mac OS X

Description

I tried running tests (OS X 10.11.3), but I get a lot of "LEAKED" errors: 

/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
Running test './apitest' under valgrind
Running tests with backend "none"...
Running test: stubdb7... LEAKED 16 BYTES
Running test: defaultctor1... LEAKED 112 BYTES
Running test: latlongcoords1... LEAKED 36 BYTES
Running test: overload1... LEAKED 72 BYTES
Running test: serialise_query1... LEAKED 132 BYTES
/Users/kronuz/Development/Kronuz/Xapian/xapian/xapian-core/tests/.libs/apitest backend none: 85 tests passed, 5 failed.
Running tests with backend "inmemory"...
Running test: collapsekey2... SKIPPED
Running test: uuid1... SKIPPED
Running test: metadata5... SKIPPED
Running test: adddoc5... SKIPPED
Running test: adddoc6... SKIPPED
Running test: nomoredocids1... SKIPPED
Running test: termtoolong1... SKIPPED
Running test: valueweightsource5... SKIPPED
/Users/kronuz/Development/Kronuz/Xapian/xapian/xapian-core/tests/.libs/apitest backend inmemory: All 256 tests passed, 8 skipped.
Running tests with backend "glass"...
Running test: collapsekey2... SKIPPED
Running test: stubdb2...dyld: lazy symbol binding failed: Symbol not found: __ZN6Xapian6Remote4openERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_jiS9_
  Referenced from: /Users/kronuz/Development/Kronuz/Xapian/xapian/xapian-core/.libs/libxapian-1.3.6.dylib
  Expected in: flat namespace

dyld: Symbol not found: __ZN6Xapian6Remote4openERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_jiS9_
  Referenced from: /Users/kronuz/Development/Kronuz/Xapian/xapian/xapian-core/.libs/libxapian-1.3.6.dylib
  Expected in: flat namespace

/bin/sh: line 1: 84692 Trace/BPT trap: 5       ./runtest ${dir}$tst
FAIL: apitest
Running test './internaltest' under valgrind
Running test: exception1... LEAKED 16 BYTES
FAIL: internaltest
Running test './stemtest' under valgrind
The random seed is 42
Please report the seed when reporting a test failure.
Running tests with arabic stemmer...
Running test: stemdict... SKIPPED
Running tests with armenian stemmer...
Running test: stemdict... SKIPPED
Running tests with basque stemmer...
Running tests with catalan stemmer...
Running test: stemdict... SKIPPED
Running tests with danish stemmer...
Running tests with dutch stemmer...
Running tests with earlyenglish stemmer...
Running test: stemdict... SKIPPED
Running tests with english stemmer...
Running tests with finnish stemmer...
Running tests with french stemmer...
Running tests with german stemmer...
Running tests with german2 stemmer...
Running tests with hungarian stemmer...
Running tests with italian stemmer...
Running tests with kraaij_pohlmann stemmer...
Running tests with lovins stemmer...
Running tests with norwegian stemmer...
Running tests with porter stemmer...
Running tests with portuguese stemmer...
Running tests with romanian stemmer...
Running tests with russian stemmer...
Running tests with spanish stemmer...
Running tests with swedish stemmer...
Running tests with turkish stemmer...
PASS: stemtest
Running test './queryparsertest' under valgrind
Running test: queryparser1... LEAKED 128 BYTES
Running test: qp_value_range3... LEAKED 108 BYTES
FAIL: queryparsertest
Running test './termgentest' under valgrind
Running test: tg_spell2... LEAKED 16 BYTES
FAIL: termgentest
Running test './unittest' under valgrind
Running test: simple_exceptions_work1... LEAKED 16 BYTES
Running test: sortableserialise1... FAILED
Number: -inf
String:
Number: -inf
String:
Number: -1.79769e+308
String:

FAIL: unittest
========================================
5 of 6 tests failed
Please report to https://xapian.org/bugs
========================================

Change History (14)

comment:1 by Olly Betts, 8 years ago

Component: Other → Test Suite
Operating System: All → Mac OS X

There's more than just leak-related failures in there.

Do the tests pass without valgrind, e.g.: 

make check VALGRIND=

comment:2 by German M. Bravo, 8 years ago

Nope, that won't compile. it says: 

harness/backendmanager.cc:28:11: fatal error: 'valgrind/memcheck.h' file not found
# include <valgrind/memcheck.h>

comment:3 by Olly Betts, 8 years ago

Version: → git master

Kronuz said on IRC that "I did make clean and then that make check VALGRIND=". While I'm not entirely sure why that would cause a failure to find the valgrind header, what I hoping to try was just running make check VALGRIND= after make check had failed as above.

What I'm particularly wanting to see is if disabling use of valgrind when running the tests fixes the sortableserialise1 failure.

The symbols dyld is failing to find look suspiciously like the ones you changed in an ABI incompatible way in the patch for #711 - are you perhaps trying to run tests built against a stock libxapian but actually using your patched build at run-time?

comment:4 by German M. Bravo, 8 years ago

Resolution: → invalid
Status: new → closed

I've verified this, tests are all working is OS X when I disable Valgrind.

comment:5 by Olly Betts, 8 years ago

Resolution: invalid
Status: closed → reopened
Summary: Tests in OS X fail → valgrind leak checking false positives on OS X

The most recent valgrind release seems to be 3.11.0:

​http://valgrind.org/docs/manual/dist.news.html

which apparently has preliminary support for OS X 10.11:

There is also preliminary support for X86/MacOSX 10.11, AMD64/MacOSX 10.11 and TILEGX/Linux.

So perhaps problems are to be expected.

My guess is that the suppressions file probably needs to be updated for each OS X release - this blog post talks about needing to create your own for 10.10:

​http://kalapun.com/posts/checking-c-code-with-valgrind-on-yosemite/

I think it's worth having an open ticket for the issue that valgrind leak checking can give false positives on recent OS X versions - it's something we should at least document, and ideally workaround - e.g. by providing a suppressions file (if there's one in valgrind's VCS we can probably just use that), or by disabling leak checking automatically.

comment:6 by Olly Betts, 8 years ago

Valgrind bug tracking support for OS X 10.11:

​https://bugs.kde.org/show_bug.cgi?id=348909

comment:7 by Olly Betts, 8 years ago

Cc: James Aylett added
Summary: valgrind leak checking false positives on OS X → valgrind leak checking false positives on OS X 10.11

Looking through the currently open bugs blocking that valgrind bug, I no longer see any leak checking ones open.

Can someone with OS X 10.11 check if the testsuite now passes under valgrind? If it does, what valgrind version are you using?

comment:8 by James Aylett, 8 years ago

So valgrind hasn't released a new version…under 3.11.0 I'm seeing qp_range3 on remotetcp_chert fails (which it doesn't do normally), and sortableserialise1 fail still (same error). Plus some leaks. 

Running test: qp_range3... QueryParserError: Unknown range operation           
Query: -10..-10
Query: -10..-9.5
Query: -10..-9
Query: -10..-8.5

Running make check VALGRIND= passes everything.

comment:9 by Olly Betts, 7 years ago

Valgrind 3.12 is now out and has "preliminary support for X86/MacOSX 10.11/12, AMD64/MacOSX 10.11/12": ​http://valgrind.org/docs/manual/dist.news.html

But no useful action shows up on ​https://bugs.kde.org/show_bug.cgi?id=348909 and nothing in the changelog for 3.12 seems to match the issues we're seeing.

comment:10 by Olly Betts, 6 years ago

Valgrind 3.13 is now out and "Preliminary support for MacOS 10.12 (Sierra) has been added."

How does the testsuite now look with valgrind on 10.11 and/or 10.12?

comment:11 by Olly Betts, 5 years ago

Using travis-ci (by just adding valgrind to the list of homebrew packages to install) I tested on macOS 10.13.3 with valgrind 3.14.0 and there are multiple issues reported for the first testcase tried (which is not a complex testcase): 

Running test './apitest' under valgrind
warning: no debug symbols in executable (-arch x86_64)
warning: no debug symbols in executable (-arch x86_64)
Running tests with backend "honey"...
Running test: zerodocid1...==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AE00: bcc_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AEA4: df_bcc_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AA8D: df (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A6A9: nistctr_init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A0F3: init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027E5A: arc4_init (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x101027F78: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AE00: bcc_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AF34: df_bcc_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AA8D: df (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A6A9: nistctr_init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A0F3: init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027E5A: arc4_init (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x101027F78: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AE00: bcc_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7AAE6: df (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A6A9: nistctr_init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A0F3: init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027E5A: arc4_init (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x101027F78: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7ABA8: df (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A6A9: nistctr_init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A0F3: init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027E5A: arc4_init (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x101027F78: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E7AD34: drbg_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A722: nistctr_init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A0F3: init (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027E5A: arc4_init (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x101027F78: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A296: generate (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027FE3: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100849648: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E76F86: ccaes_vng_ctr_crypt (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7ACB7: drbg_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A2E1: generate (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027FE3: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E7ACD4: drbg_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A2E1: generate (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027FE3: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100849648: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Conditional jump or move depends on uninitialised value(s)
==33750==    at 0x100E7AD34: drbg_update (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x100E7A2E1: generate (in /usr/lib/system/libcorecrypto.dylib)
==33750==    by 0x101027FE3: arc4random_buf (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x1010130CC: uuid_generate_random (in /usr/lib/system/libsystem_c.dylib)
==33750==    by 0x10084FE9D: Uuid::generate() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100849648: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Syscall param write(buf) points to uninitialised byte(s)
==33750==    at 0x1012F79D2: write (in /usr/lib/system/libsystem_kernel.dylib)
==33750==    by 0x100AAABD0: io_write(int, char const*, unsigned long) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10093084C: GlassVersion::write(unsigned int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975F6: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100849648: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10053AA20: BackendManagerHoney::do_get_database_path(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) (in /Users/travis/build/ojwb/xapian/xapian-core/tests/.libs/apitest)
==33750==    by 0x10050E1A4: BackendManager::do_get_database(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::_s/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975F6: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== Syscall param write(buf) points to uninitialised byte(s)
==33750==    at 0x1012F79D2: write (in /usr/lib/system/libsystem_kernel.dylib)
==33750==    by 0x100AAABD0: io_write(int, char const*, unsigned long) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10093084C: GlassVersion::write(unsigned int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10089AE5D: GlassDatabase::set_revision_number(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10089F7BC: GlassDatabase::apply() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A29FD: GlassWritableDatabase::apply() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2900: GlassWritableDatabase::commit() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10072965C: Xapian::WritableDatabase::commit() (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x10053AA8C: BackendManagerHoney::do_get_database_path(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) (in /Users/travis/build/ojwb/xapian/xapian-core/tests/.libs/apitest)
==33750==    by 0x10050E1A4: BackendManager::do_get_database(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) (in /Users/travis/build/ojwb/xapian/xapian-core/tests/.libs/apitest)
==33750==    by 0x10050E291: BackendManager::get_database(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (in /Users/tra.dylib)
==33750==    by 0x100F07AB3: libdispatch_init (in /usr/lib/system/libdispatch.dylib)
==33750==    by 0x100E179C2: libSystem_initializer (in /usr/lib/libSystem.B.dylib)
==33750==    by 0x100671A09: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==33750==    by 0x100671C39: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==33750== 
==33750== 104 (32 direct, 72 indirect) bytes in 1 blocks are definitely lost in loss record 73 of 96
==33750==    at 0x1007042FE: malloc_zone_malloc (in /usr/local/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
750==    by 0x100931517: GlassVersion::create(unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008975BE: GlassDatabase::create_and_open_tables(int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100896473: GlassDatabase::GlassDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, unsigned int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A236D: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x1008A2538: GlassWritableDatabase::GlassWritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100845596: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750==    by 0x100849648: Xapian::WritableDatabase::WritableDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) (in /Users/travis/build/ojwb/xapian/xapian-core/.libs/libxapian-1.5.0.dylib)
==33750== 
==33750== 3,264 bytes in 51 blocks are definitely lost in loss

It seems that valgrind support for recent macOS is too poor to be automatically enabling, and this isn't some transient situation as it seems to have been problematic for close to 3 years, perhaps more.

I'm thinking we should just adjust configure.ac so that on macOS the default is to not use valgrind (and people can use ./configure VALGRIND=valgrind if the really want to enable it).

Perhaps we should change that default for all platforms - ubsan and asan cover much of what valgrind can do with significantly less overhead. It's annoying that memsan is hard to use for C++ code (you need to recompile the standard C library with memsan for it to work usefully). The main thing we get with valgrind alone is per test-case leak checking due to integration into our test harness. Most such leaks will be the same on macOS and Linux at least.

comment:12 by Olly Betts, 5 years ago

I got travis to try with valgrind on all the macOS versions they support.

Both 10.13.3 and 10.12.6 fail with valgrind errors. On 10.10 the build times out (but travis warns that's deprecated and will be removed in "January 2019" - it's now March!)

Given it doesn't seem to work on any macOS version and that nobody's argued against it, I'm going to look at disabling automatic use of VALGRIND on macOS as I proposed earlier.

comment:13 by Olly Betts, 5 years ago

Milestone: → 1.4.12

Implemented in git master in [7cad6bb7bb021f0d4639a7cbc0b3b0ea0b87012e].

Marking for 1.4.12.

comment:14 by Olly Betts, 5 years ago

Resolution: → fixed
Status: reopened → closed

Backported in [8c72a4013868d33eeee125a544712124c2fe6f59].

Note: See TracTickets for help on using tickets.