Linking with OpenFace libraries

[funkmeisterb]

Hi,

I am on macOS 10.13 and I managed to build and run the libraries with examples. I can run FaceLandmarkVid and FaceLandmarkVidMulti, and it works. I am working with the master branch.

Now I want to replicate some of the behaviour of these sample apps in another C++ application (with OpenFrameworks), so I am including the libraries and linking statically.

My single-face example works just fine, based on FaceLandmarkVid.
I then modified the code to support multiple faces based on FaceLandmarkVidMulti, and this is where the problem happened.

The problem is that FaceAnalyserParameters::getModelLoc() returns the empty string.
My OpenFrameworks logic is similar to https://github.com/TadasBaltrusaitis/OpenFace/blob/master/exe/FaceLandmarkVidMulti/FaceLandmarkVidMulti.cpp. Consider line 158, "face_analysis_params.OptimizeForImages();".

Based on https://github.com/TadasBaltrusaitis/OpenFace/blob/588285753c695847ad6cdcaed958f4b2ac152d0f/lib/local/FaceAnalyser/src/FaceAnalyserParameters.cpp, function OptimizeForImages() should set the value of member attribute model_location. When I run my app, I get the following error

Reading the AU analysis module from: �
Couldn't open the model file, aborting

This means that FaceAnalyserParameters::getModelLoc() returns the empty string, but that is clearly no the case. I have even managed to modify the content of FaceAnalyserParameters::OptimizeForImages() to log debug info and understand what is going on, and I see that the first if condition is matched, so "this->model_location = model_path.string();" is run, but if I print these two strings immediately, I get the empty string!

What is going on? Any special care I should take when linking with the OpenFace libraries?
For the record, I can run FaceLandmarkVidMulti and see that the similar calls work properly in the sample app.

One important thing to note is that OpenFrameworks 0.9.8 uses boost 1.58 (in the version I have) and I built OpenFace with boost 1.60. I don't expect much difference, and the versions seem close enough.

[TadasBaltrusaitis]

Are you running the code from a different location? The code looks for the data in current directory, working directory, and executable directory. When initializing FaceAnalyserParameters you can pass it the root directory for data (the one that contains AU_predictors and model folders)

It is especially strange as the FaceAnalyserParameters is used in the same way in a single image and multiple image case, with a small exception of OptimizeForImages(), what happens if you do not call OptimizeForImages?

There is a small chance that boost might be causing this issue, what is model_path set to before the if statements?

[funkmeisterb]

Hi, thanks for your reply.

I am running the code from a different location, because I am building a OpenFrameworks app and it runs in a separate directory from the OpenFace code.

Not calling OptimizeForImages changes nothing.

I have added more code to FaceAnalyserParameters::init() and here is what it looks like:

if (boost::filesystem::exists(model_path))
	{
        std::cout << "init - Model location before is '" << this->model_location << "'" << std::endl;
        std::cout << "init - Model location matches condition 1" << std::endl;
        std::cout << "init - Model path is '" << model_path.string() << "'" << std::endl;
		this->model_location = model_path.string();
        std::cout << "init - Model location is now '" << this->model_location << "'" << std::endl;
	}

Here is what I see in my app's console:

init - Model location before is 'AU_predictors/main_dynamic_svms.txt'
init - Model location matches condition 1
init - Model path is '�'
init - Model location is now '�'

Here is what I see when I run FaceLandmarkVidMulti with the same code

init - Model location before is 'AU_predictors/main_dynamic_svms.txt'
init - Model location matches condition 1
init - Model path is 'AU_predictors/main_dynamic_svms.txt'
init - Model location is now 'AU_predictors/main_dynamic_svms.txt'

So it looks like command boost::filesystem::path model_path = boost::filesystem::path(this->model_location); returns an empty path even though this->model_location is not empty, and that condition is matched because the path '' (empty string) "exists".

This really is a boost issue with the constructor. I guess my next step is to try to force OpenFace's boost version into my OpenFrameworks app. Using different versions does not seem like a good idea.

I'm also starting to think the strings are not empty, but a different encoding is the reason why I see character � pop up after copy-pasting the result.

[funkmeisterb]

BTW I noticed that another library of OpenFace works fine with the paths I send to it.
CLNF::CLNF(string fname) in the LandmarkDetector library uses the string properly to find what it needs, and it also uses boost.

[funkmeisterb]

Building the libraries with Boost 1.57 fixed the issue. Thanks to @ofZach for the fix.
antimodular/ofxOpenFace#1