Improved Tcl smoketest

[Michael Schlenker]

The Tcl8 smoketest.tcl has some shortcomings:

1. A null bytes is not encoded as \0, the correct form is either \x00 or \u0000

(the \u0000 one is better, as the \x is too greedy) Or use [binary format] or [encoding convertto] to create binary data.

2. There is no test if the Tcl package version (given by [package provide

xapian] after the load) is matching the version used in the lib.

3. It could be re-written using the standard tcl tcltest package.

(see ​http://www.tclscripting.com/articles/apr06/article1.html for an intro)

[Michael Schlenker]

Improved smoketest.tcl file

[Olly Betts]

1. Using "\0" seems to work identically (tcl 8.4):

$ echo 'puts "\0"' | tclsh | xxd 0000000: 000a .. $ echo 'puts "\x00"' | tclsh | xxd 0000000: 000a .. $ echo 'puts "\u0000"' | tclsh | xxd 0000000: 000a ..

But if using "\0" is undesirable, I'm happy to change it.

"\u0000" isn't recognised by tcl 8.0, which doesn't matter much but seems a slight reason to prefer "\x00" perhaps? Also, we really do want a zero byte, not unicode character zero so semantically it's perhaps more appropriate.

2. The Tcl package version may not match the C++ library's Xapian

version. For example if 0.9.7 doesn't involve any ABI changes, then the 0.9.6 bindings, linked against 0.9.6, could end up running against 0.9.7.

This is very similar to XAPIAN_MINOR_VERSION (a preprocessor #define) and Xapian::minor_version() in C++ - the reason both are there is they may not be the same.

3. I don't have any great objections to using standard test harnesses for

the different languages, though there's some advantage to keeping the various smoketest versions in a similar form for ease of adding new tests to them all. Also I don't really have the time to learn the test harnesses for multiple languages. But if you think there are benefits from translating the tests and that it'll be obvious enough to me how to add a new testcase, I'm happy to accept patches.

[Olly Betts]

​http://tmml.sourceforge.net/doc/tcl/Tcl.html says:

[8] Backslash substitution. [...]

\ooo

The digits ooo (one, two, or three of them) give an eight-bit octal value for the Unicode character that will be inserted. The upper bits of the Unicode character will be 0.

So \0 is just a special case of this (where there's one octal digit which is a zero).

Is it better in some sense to use \x00 or \u0000? Or should I just leave this as it is?

[Michael Schlenker]

Your right that \0 is just a special case. But i checked with the Tcl Core maintainers about this NULL byte issue and looked at the code created by SWIG for the wrapper and have more or less good news for you about this:

There can _NEVER_ be a real null byte be passed into that function by that swig wrapper function. All NULLs are converted to 0xc080 (denormalized utf-8), so no NULL will ever reach the xapian lib from that side.

Unless the code deliberatly uses the tcl library testing code to create an invalid Tcl_Obj or you use a completly different wrapper based on Tcl_GetByteArrayFromObj() instead of the Tcl_GetStringFromObj() in use inside the current swig wrapper.

So the point is: Unless you create an invalid Tcl object deliberatly, there is no way a NULL byte can enter that function. The test does not pass a NULL byte in any case, if you use \0 or \u0000 or anything else.

[Olly Betts]

Upon further reflection, I don't really think we actually have a problem here. While the smoketest zero byte test doesn't check quite the same thing as the analagous checks in other languages do, it does at least test that a Tcl string with a NUL can be transparently stored in Xapian and retrieved. Since you can't actually create a Tcl string with a "real" zero byte in, it's a moot point whether you can pass such a string to Xapian and back!

I've added a comment noting what the smoketest is actually testing here, so I'll close this bug. Thanks for raising the issue.