Experhash-0.1.1/priv/experhash_port: Syntax error: word unexpected (expecting ")")

jpinnix · April 16, 2019, 11:48am

I’m using experhash to do perception hashing on images. It is working fine locally, but in production I am getting an error. It requires Magick++, the ImageMagick C++ API, so you have to mix compile it. I’m using Distillery 2 and Docker to make releases on Ubuntu 16.04. Imagemagick is installed on both the docker image and on the production server, and mix compile runs during the build.

Genserver times out, but I believe that is just a symptom of the real issue with experhash.

experhash-0.1.1/priv/experhash_port: Syntax error: word unexpected (expecting ")")

Phoenix 1.4.3
Elixir 1.8.1
Distillery 2.0 using the new Elixir provider

11:07:51.097 request_id=FZXvw-ylKvX9vlEAAADx [debug] Processing with DistanceWeb.DocumentController.new/2
  Parameters: %{}
  Pipelines: [:browser]
11:07:51.098 request_id=FZXvw-ylKvX9vlEAAADx [info] Sent 200 in 3ms
11:08:02.951 request_id=FZXvxq9fFd2k4d0AAAEB [info] POST /documents
11:08:02.955 request_id=FZXvxq9fFd2k4d0AAAEB [debug] Processing with DistanceWeb.DocumentController.create/2
  Parameters: %{"_csrf_token" => "RysAfnR6Yk4BenoDIB0KAgxlDDU8JgAA3n//DH7xH77LcEr0EP8EVQ==", "_utf8" => "✓", "document" => %{"file" => %Plug.Upload{content_type: "image/png", filename: "a028.png", path: "/tmp/plug-1555/multipart-1555412882-15036320304116-1"}}}
  Pipelines: [:browser]
/home/deploy/lib/experhash-0.1.1/priv/experhash_port: 1: /home/deploy/lib/experhash-0.1.1/priv/experhash_port: Syntax error: word unexpected (expecting ")")
11:08:12.959 request_id=FZXvxq9fFd2k4d0AAAEB [info] Converted exit {:timeout, {GenServer, :call, [#PID<0.1680.0>, {:command, {:hash, :dd, "media/1555412882-15036320304116-1-a028-orig.png", 8}}, 10000]}} to 500 response
11:08:12.961 request_id=FZXvxq9fFd2k4d0AAAEB [error] Process #PID<0.1678.0> terminating
** (exit) an exception was raised:
    ** (ErlangError) Erlang error: {{:timeout, {GenServer, :call, [#PID<0.1680.0>, {:command, {:hash, :dd, "media/1555412882-15036320304116-1-a028-orig.png", 8}}, 10000]}}, {DistanceWeb.Endpoint, :call, [%Plug.Conn{adapter: {Plug.Cowboy.Conn, :...}, assigns: %{}, before_send: [], body_params: %Plug.Conn.Unfetched{aspect: :body_params}, cookies: %Plug.Conn.Unfetched{aspect: :cookies}, halted: false, host: "redacted", method: "POST", owner: #PID<0.1678.0>, params: %Plug.Conn.Unfetched{aspect: :params}, path_info: ["documents"], path_params: %{}, port: 80, private: %{}, query_params: %Plug.Conn.Unfetched{aspect: :query_params}, query_string: "", remote_ip: {127, 0, 0, 1}, req_cookies: %Plug.Conn.Unfetched{aspect: :cookies}, req_headers: [{"accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"}, {"accept-encoding", "gzip, deflate, br"}, {"accept-language", "en-US,en;q=0.5"}, {"connection", "upgrade"}, {"content-length", "30026"}, {"content-type", "multipart/form-data; boundary=---------------------------7719490031198181894888934539"}, {"cookie", "_distance_key=SFMyNTY.g3QAAAABbQAAAAtfY3NyZl90b2tlbm0AAAAYdEUvUTAyVTZJTU1PQ1h4Mkk1NHBqdz09.Ojpdcrhb9AN8xoU4Sjxl5rR-uCsb6wFd-FJlt6vuz4Y"}, {"host", "redacted"}, {"referer", "https://redacted/documents/new"}, {"upgrade-insecure-requests", "1"}, {"user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Firefox/68.0"}, {"x-cluster-client-ip", "97.89.62.78"}, {"x-forwarded-for", "97.89.62.78"}], request_path: "/documents", resp_body: nil, resp_cookies: %{}, resp_headers: [{"cache-control", "max-age=0, private, must-revalidate"}], scheme: :http, script_name: [], secret_key_base: nil, state: :unset, status: nil}, []]}}
(phoenix) lib/phoenix/endpoint/cowboy2_handler.ex:49: Phoenix.Endpoint.Cowboy2Handler.init/2
(cowboy) /redacted/distance/deps/cowboy/src/cowboy_handler.erl:41: :cowboy_handler.execute/2

OvermindDL1 · April 16, 2019, 3:34pm

Are you actually compiling it ‘within’ the same docker image? If not and if there are platform differences, like say the shells that are being used in the compilation, that would cause that error.

My gut instinct is that it’s missing ‘bash’ in the image or something, so would need to install that.

jpinnix · April 16, 2019, 5:42pm

@OvermindDL1 I think that Distillery copied over the deps which included this compiled version.

So I ran mix deps.clean experhash , mix deps.get, and with mix deps.compile I was finally able to see the error in compilation.

The issue is with the Makefile apparently.

@OvermindDL1 Do you think I should make a separate post about the Makefile issue as it involves Erlang too and isn’t really a docker issue?

OvermindDL1 · April 16, 2019, 5:51pm

As it is still an experhash issue I’d wager that the makefile bug with experhash should go here too.

That can be an issue if something is compiling in the deps folder instead of into the _build folder like it is supposed to, because if it compiles for ‘your’ system, say via dev, then you compile for the production system, then you can get mismatched all kinds of things. That is why I always freshly git clone into the docker build image, of which I then build it inside there and have distillery make a release, then I have docker prune the image into a new one that contains just the released directory for running.

jpinnix · April 16, 2019, 7:05pm

Here are the errors in the log:

deploy@distance:~/distance$ mix deps.compile
===> Compiling ranch
===> Compiling telemetry
==> experhash
Makefile:21: cpp/src/decoder.d: No such file or directory
Makefile:21: cpp/src/experhash_port.d: No such file or directory
Makefile:21: cpp/src/encoder.d: No such file or directory
Makefile:21: cpp/src/hash.d: No such file or directory
g++ -std=c++11 -g -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/lib/erlang/usr/include -L/usr/lib/erlang/usr/lib   -c -o cpp/src/decoder.o cpp/src/decoder.cpp
g++ -std=c++11 -g -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/lib/erlang/usr/include -L/usr/lib/erlang/usr/lib   -c -o cpp/src/experhash_port.o cpp/src/experhash_port.cpp
g++ -std=c++11 -g -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/lib/erlang/usr/include -L/usr/lib/erlang/usr/lib   -c -o cpp/src/encoder.o cpp/src/encoder.cpp
g++ -std=c++11 -g -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/lib/erlang/usr/include -L/usr/lib/erlang/usr/lib   -c -o cpp/src/hash.o cpp/src/hash.cpp
g++ -std=c++11 -g -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/lib/erlang/usr/include -L/usr/lib/erlang/usr/lib -o priv/experhash_port cpp/src/decoder.o cpp/src/experhash_port.o cpp/src/encoder.o cpp/src/hash.o -lei -L/usr/local/lib -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI

and here is the Makefile itself:

erlang_base = $(shell erl -eval 'io:format("~s", [code:root_dir()])' -s init stop -noshell)/usr

CXXFLAGS += -std=c++11 -g $(shell pkg-config --cflags Magick++) -I$(erlang_base)/include -L$(erlang_base)/lib
LDFLAGS += -lei $(shell pkg-config --libs Magick++)

sources = $(wildcard cpp/src/*.cpp)
objects = $(sources:.cpp=.o)
header_deps = $(sources:.cpp=.d)

priv/experhash_port: $(objects) | priv
	$(CXX) $(CXXFLAGS) -o $@ $^ $(LDFLAGS)

priv:
	mkdir -p $@

cpp/src/%.d: cpp/src/%.cpp
	@set -e; \
	rm -f $@; \
	$(CXX) -MM -MT '$(@:.d=.o) $@' -MF $@ $(CPPFLAGS) $(CXXFLAGS) $<

include $(header_deps)

.PHONY: clean
clean:
	rm -f priv/experhash_port $(objects) $(header_deps)

So, it seems to be a path issue: Makefile:21: cpp/src/decoder.d: No such file or directory. Line 21 in the Makefile is include $(header_deps). The files are in the location specified.

Oh, and make on the server is 4.1 and on my Mac is 3.81.

OvermindDL1 · April 16, 2019, 7:22pm

Hmm, so what directory is it in then, put $(info >$(CURDIR)<) on the line immediately before the include $(header_deps) line and check?

That’s pretty ancient! ^.^;
Though unrelated I’d think? Hmm…

NobbZ · April 16, 2019, 7:23pm

Those *.d files are not necessary to build the tool, its only that they make incremental builds on changes a bit faster if generated.

Their absence should not influence the build itself.

OvermindDL1 · April 16, 2019, 7:27pm

Hmm, except the include is supposed to include those files as makefiles in the location where include is called and if the file is missing then it is a hard error. The -include command should be used if it is an optional include and not a hard required include that include does (do note that it can autobuild them if there are rules to build them). And as it is top level then they are definitely hard required. A few things about that Makefile seems fishy but still…

jpinnix · April 16, 2019, 8:00pm

Praise Jesus
All I did was add $(info >$(CURDIR)<) and it compiled. I mean, it doesn’t make sense as all it was supposed to do was say what the current directory is, but I’ll take it.

It’s the default on the latest macOS. Seems to work most of the time?

Thank you @OvermindDL1 and @NobbZ !

OvermindDL1 · April 16, 2019, 8:33pm

o.O??!?

That makes me wonder if there were some stray spaces or tabs in the file or so (m4 is *very whitespace specific)… ^.^;

Ooor, I wonder if it slowed it down ‘enough’ to let the files be created somewhere else, might be an issue of the filesystem being used, hmm…

Lol, that’s a very weird issue though, I’d still report it to the dep dev’s! ^.^

jpinnix · April 18, 2019, 3:45am

I had already added an issue, but saw the dev say on another issue that he isn’t really doing anything with it anymore.

OvermindDL1 · April 22, 2019, 3:18pm

That sounds like time to Fork it then?

kemonomachi · April 25, 2019, 5:18pm

I’m not working actively with this project, but I still want to fix problems like this. I’ll take a look at the Makefile and see if I can improve it. I’ve written less than a handful of Makefiles ever, so I’m not surprised that it is a bit wonky.

kemonomachi · April 27, 2019, 9:18am

The Makefile:21: cpp/src/decoder.d: No such file or directory lines are warnings from make that will show up when compiling from scratch with the current Makefile. A bit annoying, but should be harmless. In the output you can see that make continues and actually compiles everything properly. I’ll see if I can get rid of the warning, but this is not the root cause.

I think the problem is that I compile the C++ code straight into the main directory instead of using the _build directory, as OvermindDL1 mentioned above. If this then is copied over, make will think the executable is up-to-date and skip building it.

@jpinnix, does it work if you run mix deps.clean experhash before creating a release? I’m not familiar with how Distillery works, but if you can prevent it from copying over any experhash build artifacts I think it should build OK inside the Docker image.

jpinnix · May 3, 2019, 7:18pm

Huge thanks to @kemonomachi for updating Experhash to use elixir_make. Fixes all of the issues.