gettext is broken (no such file textdomain_manager)

[rriemann]

I'm the maintainer of kdegettext.rb, which builds on top of gettext. After the recent changes of rubygems, the gettext gem seems to be completely broken. Looks like you already started to fix these issues. Sincerely your current master doesn't work for me either.

It would be really important to get a working version again. Looking at the network graph of mutoh's version, shows off, that there a many forks, but nothing got merged into the original gem version.

The initial idea was to use kdegettext and gettext in the KDE automated internationalization tool chain hooked into our git hosting system. So I hope to see a working gettext again - maybe a version 2.2.0?

rgettext
/home/rriemann/.rvm/gems/ruby-1.9.2-p290/gems/gettext-2.0.0/bin/rgettext:19:in `rescue in rescue in <top (required)>': Ruby-GetText-Package are not installed. (RuntimeError)
        from /home/rriemann/.rvm/gems/ruby-1.9.2-p290/gems/gettext-2.0.0/bin/rgettext:15:in `rescue in <top (required)>'
        from /home/rriemann/.rvm/gems/ruby-1.9.2-p290/gems/gettext-2.0.0/bin/rgettext:12:in `<top (required)>'
        from /home/rriemann/.rvm/gems/ruby-1.9.2-p290/bin/rgettext:19:in `load'
        from /home/rriemann/.rvm/gems/ruby-1.9.2-p290/bin/rgettext:19:in `<main>'

Doesn't work. So let's try it interactively:

irb
ruby-1.9.2-p290 :001 > require 'gettext'
LoadError: no such file to load -- gettext/runtime/textdomain_manager
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from /home/rriemann/.rvm/gems/ruby-1.9.2-p290/gems/gettext-2.0.0/lib/gettext.rb:32:in `<top (required)>'
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:59:in `require'
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:59:in `rescue in require'
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
        from (irb):1
        from /home/rriemann/.rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `<main>'
ruby-1.9.2-p290 :002 >

[cameel]

Hi. Sorry for a late response.

I didn't have much time for this fork lately, but I'll look into it. Can't promise anything soon though - maybe later this week.

I'm not sure I will be making stable releases - I probably won't have time to actively maintain it. This gem breaks for me quite often. I'd prefer mutoh to pull the patches into the original repo, but it does not seem likely.

BTW: The fact that this gem was totally broken when I created the fork and there was no working one for Rails 3 makes me wonder if anyone is using gettext in Ruby at all. There's fast_gettext, but it does not have anything to extract strings from the code. Maybe there's some obscure tool for creating .pot files from Ruby that everyone's using?

[rriemann]

Hi Kamil!

Am Dienstag, 22. November 2011, 19:50:55 schrieben Sie:

Hi. Sorry for a late response.

I didn't have much time for this fork lately, but I'll look into it. Can't
promise anything soon though - maybe later this week.

Even in the next few weeks would be awesome. Better late than never. I would
do it myself, but I'm not expert enough do understand why it isn't working. :(

I'm not sure I will be making stable releases - I probably won't have time
to actively maintain it. This gem breaks for me quite often. I'd prefer
mutoh to pull the patches into the original repo, but it does not seem
likely.

I had already last year a mail conversation with mutoh. Maybe you are right.
Gettext doesn't seem to belong to his priorities. I would support your merge
request by some motivating comments. Just drop me a note when you did it! ;)

BTW: The fact that this gem was totally broken when I created the fork and
there was no working one for Rails 3 makes me wonder if anyone is using
gettext in Ruby at all. There's fast_gettext, but it does not have anything
to extract strings from the code. Maybe there's some obscure tool for
creating .pot files from Ruby that everyone's using?

I have also the impression, that the standard toolchain gettext with po and
pot files isn't so popular in ruby-world. Nevertheless it is THE choice for
KDE applications. We have an automated infrastructure to extract strings from
code, deliver them to translators and submit them back into the repo again.
There are also nice desktop application to help translaters doing their job:
http://kde.org/applications/development/lokalize/ (linux, mac, windwows?)

In the ruby world there are a lot of other possibilities for i18n.

KDE is also available for python and php. Luckily they don't suffer the same
problem, as GNU gettext offers natively support for them.

The supported languages are C, C++, ObjectiveC, PO, Python, Lisp,
EmacsLisp, librep, Scheme, Smalltalk, Java, JavaProperties, C#, awk, YCP,
Tcl, Perl, PHP, GCC-source, NXStringTable, RST, Glade.

Clearly ruby is missing in this list. So maybe it would be the best to just
put ruby support in GNU gettext. Unfortunately I'm using some ruby gems like
sexp_processor and ruby_parser for code analysis. Porting this to pure C might
be quite complex. What do you think?

Best,
Robert

[cameel]

I am using it for i18n in my application so it'll be fixed sooner or later. I'll let you know in this thread.

What surprises me is that there are actively maintained "fast_gettext" and "gettext_i18n_rails" gems. Someone is using gettext in Ruby. But how do they extract the strings from the code? I doubt they do this manually. I've seen a gem for reporting untranslated strings in the log file but it's for the built-in Rails i18n. I have started a thread on Ruby on Rails Talk mailing list to find out.

Support for Ruby in gettext would probably be better than in a gem, I could even try to do it, but I'm concerned with the maintenance. Would gettext people be willing to fix the code on their own when something breaks or would they just drop the unmaintained Ruby part? In the current situation where I can't really commit to the maintenance I think that a gem hosted on GitHub is a better solution. There is a lot of Ruby folk willing to submit patches here, so even if it just sits there unmaintained anyone can easily fork it and fix it for their own use.

[dgilperez]

Hi cameel,

We're in the same situation at out company. We are using gettext, fast_gettext and gettext_i18n_rails with the same workflow, using gettext to extract the strings and we get hit often with issues related to outdated and not maintained gem.

We are currently not able to use gettext with rails3, ruby 1.9.2-p290 and the lastest rubygems. Your patch works with the rake find and pack tasks, but then the application fail to show certain strings. I've commented in the original issue in mutoh's repository, mutoh#11

We'd like to keep using gettext, but we are considering moving to i18n after this issues. But if you are willing to take over, we are willing to help if we can.

[cameel]

Hi

Unfortunately I failed to find enough time for this last week and earlier this week, but since there seem to be ever more people wanting this fixed, I'll m going to look into it tomorrow and see what I can do.

[cameel]

I have started working on this today. I want to make gettext:find work first (it's not working with my application and that's one thing I can reliably reproduce). Then I'll try to fix issues mentioned in this thread.

First thing is that gettext fails in projects using Haml because gettext_i18n_rails that provides Haml parser is missing a dependency on ruby_parse tree. I have submitted a patch for this. grosser/gettext_i18n_rails#39 . A quick workaround for this is to add ruby_parser to your Gemfile:

gem 'ruby_parser'

[cameel]

Today I have fixed rake gettext:store_model_attributes to work with namespaced models: grosser/gettext_i18n_rails#40

It turned out not to be caused by broken gettext but by the fact that gettext_i18n_rails ignores namespaced models so it doesn't solve your problems but I did not know the cause when I started debugging it. Anyway, it's another step forward. More fixes coming soon.

It also turns out that my last patch to gettext_i18n_rails (already pulled in and released with gettext_i18n_rails 0.3.1) does fix rake gettext:find, at least when used with my application. It did not work yesterday - it was printing errors about using a nil - but everything was ok when I tried to debug it today. I probably forgot to do bundle install or did some similar simple mistake.

[cameel]

Today I have dug deeper into into problems with rgettext and below is what I found.

For me it fails in locale gem because gettext uses itself to translate its own messages but rgettext script does not explicitly set or detect locale. It would not normally be a problem (it should fall back to using default locale, i.e. :en), but there's a bug in locale gem that makes it fail if no locale is set. The bug has been fixed long ago, but the current version (2.0.5) is over a year old and a new one has not been released yet. It has even been reported: mutoh/locale#3

@salout: Your problem with rgettext is different. It seems like it can't find lib/gettext/runtime/textdomain_manager.rb file. Can you verify that this file is present in /home/rriemann/.rvm/gems/ruby-1.9.2-p290/gems/gettext-2.0.0/lib/gettext/runtime/?

You seem to be using gettext 2.0.0. I don't know if that's because of outdated gemspec in this repository (it was saying version is 2.0.0; I have just fixed it) or if you're actually using an old version. The latest version released by mutoh is 2.1.0. If it's the latter, please update to 2.1.0 from RubyGems or, better, try to use my fork for now. Easiest way to do this would be to install bundler gem, create a Gemfile file in the root directory of your project and then run bundle install in the command line to make it fetch all dependencies. As a bonus you can also get locale 2.0.6 this way:

# Gemfile
source 'http://rubygems.org'

gem 'gettext', '>= 2.1.0', :git => 'git://github.com/cameel/gettext.git'
gem 'locale', '>= 2.0.6', :git => 'git://github.com/mutoh/locale.git'

As for the long-term solution, I am starting to think that relying on gettext gem is not a good idea. A better one would be to extract parts related to creating/updating .po/.pot files into a separate gem and make it use fast_gettext which is actively maintained. Or maybe try to get get_pomo going since it seems to have similar scope.

[cameel]

@dgilperez I can't reproduce your problem. I tried both with my application and on a clean Rails 3.1.3 one and in both cases strings with embedded %{} are translated correctly. Can you provide more information on your setup? In particular, are you actually using gettext for fetching translations at runtime, rather than fast_gettext? Or does just including gettext in your Gemfile cause fast_gettext to behave like this?

My gettext setup in Gemfile is this (note :require => false that prevents gettext from interfering with things at runtime):

# Gemfile
...
gem 'rails-i18n'
gem 'gettext_i18n_rails', '>= 0.3.2'

group :development do
  gem 'gettext', '>= 2.1.0', :git => 'git://github.com/cameel/gettext.git', :require => false
end
...

[dgilperez]

Hi @cameel,

That's right, we're using fast_gettext. I added :require => false to gettext and upgraded fast_gettext version to the last in the repository, and things are going smooth now. Thanks !!!

Here is my setup, just in case it helps others:

# Gemfile
...
gem 'fast_gettext', :git => 'git://github.com/grosser/fast_gettext.git'
gem 'gettext_i18n_rails', :git => 'git://github.com/grosser/gettext_i18n_rails'

group :development do
  gem 'gettext', :git => 'git://github.com/cameel/gettext.git', :require => false
end

What are you using rails-i18n gem for? I assumed i18n support was default in rails3.1.

Many thanks for your efforts. We're now able to operate normally again, both with rake gettext:find and showing translations under ruby 1.9.2-p290.
But I agree with you: in the long run, get_pomo looks like an interesting alternative to managing the .po files.

[cameel]

Nice to hear that it works now.

rails-i18n contains localized date formats, day and month names, error messages and maybe some other i18n related stuff. As far as I know, Rails depends on the localization engine that makes localization possible but not on the actual localizations for specific languages so you need to declare that yourself in Gemfile. See svenfuchs/rails-i18n.

[rriemann]

Status-update:

Using this gems (https://gist.github.com/1435327) with ruby 1.8.7 makes kdegettext working again. Same gem version with ruby 1.9.2 breaks with this error: https://gist.github.com/1435341

The Gemfile with gem 'locale', '>= 2.0.6', :git => 'git://github.com/mutoh/locale.git'
doesn't work for me, as there is no released version 2.0.6. I left a comment in the existing issue thread.

Thank you very much for your ongoing efforts to make it work again!

[cameel]

So you're using latest gettext from RubyGems. It doesn't work with Ruby 1.9.2 because it contains newer RubyGems and it's not compatible with it. I have fixed it in my fork - mutoh#11 is the patch for that. That patch breaks it for older RubyGems though. A new one that works with both older and newer RubyGems is only in the master branch of my fork as I gave up hoping that the patches will get pulled in and did not make any further pull requests.

And as for the Gemfile - you're right, I forgot that the locale repo does not contain the .gemspec file. It has a rake task that generates it on the fly from the specification in Rakefile and builds a gem from that, never writing the spec to disk. I have just created a fork and added auto-generated gemspec to the repo so that you can use it in Gemfile. The code below should now work:

# Gemfile
source 'http://rubygems.org'

gem 'gettext', '>= 2.1.0', :git => 'git://github.com/cameel/gettext.git'
gem 'locale', '>= 2.0.6', :git => 'git://github.com/cameel/locale.git', :branch => 'add-gemspec'

Check if that works for you. If so, tomorrow I'll build gems out of these two repos and push them to RubyGems. I probably can't push them as gettext and locale so it'll be something like cameel/gettext and cameel/locale. This, however, will probably only work if you don't use any other gems that depend on gettext and locale (rails, for example, indirectly depends on locale) because that would cause two different versions to be pulled in and they will most likely clash in some way. At best one version may override the other but even then it's not certain which one.

[rriemann]

I already installed locale using the rake task. By now, I'm probably ok with getting it running using ruby 1.8.7

If you consider to rechain the gettext toolchain using other components, it might be better to do not put more efforts into the old kind of unmaintained tools from mutoh. Having different gems with same names could be a bit confusing. I'm ok with having these versions in your gitrepo and grapping it from there. :)

[cameel]

OK. So I'm closing this issue. I'll let you both know when I get some alternative tool for extracting translations working. Probably not all that soon, but in the coming months I'll find some time for that.

[dgilperez]

I appreciate your work Cameel. Thanks and keep us posted, please.