Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make some improvements around exception handling and documentation fixes #50

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/readme.en.html
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ <h3><a name="vc">Perl Bindings</a></h3>
$ make install</pre>
</div><!-- float -->
<p>
Users can find a Perl bindings in <kbd>bindings/perl/</kbd>, in which the wrapper was generated by <a href="http://www.swig.org/">SWIG</a>. To install the Perl bindings, run <kbd>perl Makefile.PL</kbd> and then <kbd>make install</kbd>. See also <kbd>bindings/perl/sample.pl</kbd>.
Users can find a Perl bindings in <kbd>bindings/perl/</kbd>, in which the wrapper was generated by <a href="https://www.swig.org/">SWIG</a>. To install the Perl bindings, run <kbd>perl Makefile.PL</kbd> and then <kbd>make install</kbd>. See also <kbd>bindings/perl/sample.pl</kbd>.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -134,7 +134,7 @@ <h3><a name="vc">Python Bindings</a></h3>
$ python setup.py install</pre>
</div><!-- float -->
<p>
Users can find a Python bindings in <kbd>bindings/python/</kbd>, in which the wrapper was generated by <a href="http://www.swig.org/">SWIG</a>. To install the Python bindings, run <kbd>python setup.py install</kbd>. See also <kbd>bindings/python/sample.py</kbd>.
Users can find a Python bindings in <kbd>bindings/python/</kbd>, in which the wrapper was generated by <a href="https://www.swig.org/">SWIG</a>. To install the Python bindings, run <kbd>python setup.py install</kbd>. See also <kbd>bindings/python/sample.py</kbd>.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -146,7 +146,7 @@ <h3><a name="vc">Ruby Bindings</a></h3>
$ make install</pre>
</div><!-- float -->
<p>
Users can find a Ruby bindings in <kbd>bindings/ruby/</kbd>, in which the wrapper was generated by <a href="http://www.swig.org/">SWIG</a>. To install the Ruby bindings, run <kbd>ruby extconf.rb</kbd> and then <kbd>make install</kbd>. See also <kbd>bindings/ruby/sample.rb</kbd>.
Users can find a Ruby bindings in <kbd>bindings/ruby/</kbd>, in which the wrapper was generated by <a href="https://www.swig.org/">SWIG</a>. To install the Ruby bindings, run <kbd>ruby extconf.rb</kbd> and then <kbd>make install</kbd>. See also <kbd>bindings/ruby/sample.rb</kbd>.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -160,7 +160,7 @@ <h3><a name="vc">Others</a></h3>
<li><a href="https://github.com/kmike/marisa-trie/">https://github.com/kmike/marisa-trie/</a> an alternative Cython-based pip-installable Python bindings which is faster than the above Python bindings.</li>
</ul>
</li>
<li>Node.js</li>
<li>Node.js
<ul>
<li><a href="https://github.com/jakwings/iojs-marisa-trie">https://github.com/jakwings/iojs-marisa-trie</a> a wrapper of marisa-trie</li>
</ul>
Expand Down
14 changes: 7 additions & 7 deletions docs/readme.ja.html
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ <h3><a name="vc">Perl バインディング</a></h3>
$ make install</pre>
</div><!-- float -->
<p>
<a href="http://www.swig.org/">SWIG</a> による Perl バインディングが <kbd>bindings/perl/</kbd> にあります.<kbd>perl Makefile.PL</kbd> により <kbd>Makefile</kbd> を作成し,<kbd>make install</kbd> を実行することでインストールできます.使い方については,<kbd>bindings/perl/sample.pl</kbd> を参考にしてください.
<a href="https://www.swig.org/">SWIG</a> による Perl バインディングが <kbd>bindings/perl/</kbd> にあります.<kbd>perl Makefile.PL</kbd> により <kbd>Makefile</kbd> を作成し,<kbd>make install</kbd> を実行することでインストールできます.使い方については,<kbd>bindings/perl/sample.pl</kbd> を参考にしてください.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -137,7 +137,7 @@ <h3><a name="vc">Python バインディング</a></h3>
$ python setup.py install</pre>
</div><!-- float -->
<p>
<a href="http://www.swig.org/">SWIG</a> による Python バインディングが <kbd>bindings/python/</kbd> にあります.<kbd>python setup.py install</kbd> により インストールできます.使い方については,<kbd>bindings/python/sample.py</kbd> を参考にしてください.
<a href="https://www.swig.org/">SWIG</a> による Python バインディングが <kbd>bindings/python/</kbd> にあります.<kbd>python setup.py install</kbd> により インストールできます.使い方については,<kbd>bindings/python/sample.py</kbd> を参考にしてください.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -149,7 +149,7 @@ <h3><a name="vc">Ruby バインディング</a></h3>
$ make install</pre>
</div><!-- float -->
<p>
<a href="http://www.swig.org/">SWIG</a> による Ruby バインディングが <kbd>bindings/ruby/</kbd> にあります.<kbd>ruby extconf.rb</kbd> により <kbd>Makefile</kbd> を作成し,<kbd>make install</kbd> を実行することでインストールできます.使い方については,<kbd>bindings/ruby/sample.rb</kbd> を参考にしてください.
<a href="https://www.swig.org/">SWIG</a> による Ruby バインディングが <kbd>bindings/ruby/</kbd> にあります.<kbd>ruby extconf.rb</kbd> により <kbd>Makefile</kbd> を作成し,<kbd>make install</kbd> を実行することでインストールできます.使い方については,<kbd>bindings/ruby/sample.rb</kbd> を参考にしてください.
</p>
</div><!-- subsection -->
<div class="subsection">
Expand All @@ -168,7 +168,7 @@ <h3><a name="vc">その他</a></h3>
<li><a href="https://github.com/komiya-atsushi/node-marisa-trie">https://github.com/komiya-atsushi/node-marisa-trie</a>Node.js から使うためのモジュール</li>
</ul>
</li>
</p>
</ul>
</div><!-- subsection -->
</div><!-- section -->

Expand Down Expand Up @@ -736,12 +736,12 @@ <h2><a name="compatibility">辞書の互換性</a></h2>
<div class="section">
<h2><a name="references">参考資料</a></h2>
<ul>
<li><a href="http://www.anlp.jp/proceedings/annual_meeting/2011/html/program.html">言語処理学会第 17 回年次大会(NLP2011)</a>
<li><a href="https://www.anlp.jp/proceedings/annual_meeting/2011/html/program.html">言語処理学会第 17 回年次大会(NLP2011)</a>
<ul>
<li>言語処理学会年次大会の予稿集が公開されていて,MARISA に関する論文(<a href="http://www.anlp.jp/proceedings/annual_meeting/2011/pdf_dir/F2-6.pdf">F2-6.pdf</a>)をダウンロードできます.</li>
<li>言語処理学会年次大会の予稿集が公開されていて,MARISA に関する論文(<a href="https://www.anlp.jp/proceedings/annual_meeting/2011/pdf_dir/F2-6.pdf">F2-6.pdf</a>)をダウンロードできます.</li>
</ul>
</li>
<li><a href="http://d.hatena.ne.jp/s-yata/20110310/1299777228">発表資料(Prefix/Patricia Trie の入れ子による辞書圧縮)</a>
<li><a href="https://d.hatena.ne.jp/s-yata/20110310/1299777228">発表資料(Prefix/Patricia Trie の入れ子による辞書圧縮)</a>
<ul>
<li>MARISA に関する発表資料(<a href="https://sites.google.com/site/headdythehero/cabine/2011/0311/NLP2011-F2-6.pptx?attredirects=0&amp;d=1">NLP2011-F2-6.pptx</a>, <a href="https://sites.google.com/site/headdythehero/cabine/2011/0311/NLP2011-F2-6.pdf?attredirects=0&amp;d=1">NLP2011-F2-6.pdf</a>, <a href="https://sites.google.com/site/headdythehero/cabine/2011/0311/NLP2011-F2-6-notes.pdf?attredirects=0&amp;d=1">NLP2011-F2-6-notes.pdf</a>)をダウンロードできます.</li>
</ul>
Expand Down
2 changes: 1 addition & 1 deletion include/marisa/exception.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class Exception : public std::exception {
Exception(const Exception &ex)
: std::exception(), filename_(ex.filename_), line_(ex.line_),
error_code_(ex.error_code_), error_message_(ex.error_message_) {}
virtual ~Exception() throw() {}
virtual ~Exception() noexcept;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noexcept is C++11. I seem to recall that @s-yata likes to keep this compatible with C++03, so maybe add some #if __cplusplus >= ...?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that throw() is also C++11. So I don't think this is changing the C++ level. It's also worth mentioning that throw() was deprecated in C++17 and removed in C++20. So there isn't a good reason to state that a destructor should throw exceptions.

https://en.cppreference.com/w/cpp/language/noexcept_spec

Do you still want this #if'd?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that throw() is also C++11.

It looks like it's C++98. See exception-specification: https://www.lirmm.fr/~ducour/Doc-objets/ISO+IEC+14882-1998.pdf

Do you still want this #if'd?

My opinion is totally unimportant and irrelevant for getting this merged, but it seems like either #if or raise the required C++ version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experimentally confirmed that noexcept requires c++11.

Try this with various -std args:

#if __has_feature(cxx_noexcept)
#error yes noexcept
#else
#error no noexcept
#endif

The current version of the PR doesn't compile for me:

libtool: compile:  g++ -DPACKAGE_NAME=\"marisa\" -DPACKAGE_TARNAME=\"marisa\" -DPACKAGE_VERSION=\"0.2.6\" "-DPACKAGE_STRING=\"marisa 0.2.6\"" -DPACKAGE_BUGREPORT=\"[email protected]\" -DPACKAGE_URL=\"\" -DPACKAGE=\"marisa\" -DVERSION=\"0.2.6\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -march=native -Wall -Weffc++ -Wextra -Wconversion -I../../../../include -I../../../../lib -g -O2 -DMARISA_USE_POPCNT -mpopcnt -DMARISA_USE_SSE4 -msse4 -MT mapper.lo -MD -MP -MF .deps/mapper.Tpo -c mapper.cc  -fno-common -DPIC -o .libs/mapper.o
...
../../../../include/marisa/exception.h:22:11: error: exception specification of overriding function is more lax than base version
  virtual ~Exception() noexcept;
          ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/exception:101:13: note: overridden virtual function is here
    virtual ~exception() _NOEXCEPT;
            ^
In file included from mapper.cc:13:
In file included from ../../../../lib/marisa/grimoire/io/mapper.h:6:
In file included from ../../../../include/marisa/base.h:191:
../../../../include/marisa/exception.h:22:23: error: expected ';' at end of declaration list
  virtual ~Exception() noexcept;
% g++ --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin


Exception &operator=(const Exception &rhs) {
filename_ = rhs.filename_;
Expand Down
1 change: 1 addition & 0 deletions lib/marisa/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ libmarisa_la_LIBADD = \
grimoire/vector/libvector.la

libmarisa_la_SOURCES = \
exception.cc \
keyset.cc \
agent.cc \
trie.cc
21 changes: 21 additions & 0 deletions lib/marisa/exception.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#include "marisa/exception.h"

namespace marisa {

/**
* This is defined here for the following reasons.
*
* 1. Virtual destructors are best not defined in a header to be included in many compilation units.
* Defining it here in a single compilation unit prevents multiple vtables from being generated and causing
* a compiler to be confused as to which is the true vtable, like with full link time optimization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't the linker just merge the vtables? What goes wrong?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my experience, no not really. Especially not when you use one in a shared library and then you include the same header in your application using that shared library. You get symbol conflicts. AIX is very finicky and has been vocal about name conflicts like this in the past when linking.

When you're using clang and using full link time optimization, I believe that it will collapse the vtables. Unfortunately when you try to cast a pointer to determine if the object is of a specific subclass, it may not use the collapsed table. This caused a lot of problems in our code in the past. So we use the -Wweak-vtables warning to avoid this issue in the future. That warning complains about this specific class, and we don't want to see this warning in our code.

* By reducing the number of vtable instances, you also reduce the size of your library or application.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you measure a size difference?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the specific number any more, but it was non-zero.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I'll have to try.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried on OS X. libmarisa.a is about 10k smaller when ~Exception is defined in exception.cc. The size difference goes away when it's linked, and the final static tools/ binaries are a few 10s of bytes smaller. libmarisa.dylib is also the same size.

I guess some sufficiently bad linker must exist somewhere though.

* You can see warnings around this with the -Wweak-vtables compiler option. Such issues may not be seen till runtime.
* Using the default keyword on the destructor is the same as defining it in the header.
*
* 2. Exceptions can not be thrown from destructors. When an exception is thrown from a destructor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be an explanation of why it's noexcept, not why it's defined here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Point 1 is mostly around why it's in this file. Point 2 is more about why I changed the signature. I can eliminate point 2 if you don't think it's helpful.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some source code static analysis tools complain about the throws usage on destructors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like something to be explained in the PR, not at the site of every destructor definition.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only destructor in this project with this issue.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wanted the reasoning to be located with the code so that it doesn't get reverted by someone else's changes.

* while the stack is unwinding from another thrown exception, then the program will crash instead
* of throwing the exception. See https://isocpp.org/wiki/faq/exceptions#ctor-exceptions
*/
Exception::~Exception() noexcept {}

}