Written in 2018
ALVAR is an augmented reality library from VTT in Finland. It finds printed markers in a camera image and works out their position and angle, which is what you need to draw a 3D object that sits on the marker as the camera moves.
There was once an interesting demo combining ALVAR with Kurento: it rendered 3D models on detected ALVAR markers in real-time WebRTC video streams, with Irrlicht, an open source 3D engine, doing the drawing. The demo had stopped building years ago. In this post we bring it back to life, as the base for a remote support prototype where a technician can point at real objects in a customer's video.
Installing ALVAR's dependencies
ALVAR needs OpenCV, GLUT and OpenSceneGraph development packages:
sudo apt-get install libopencv-dev freeglut3-dev libopenscenegraph-devCompiling ALVAR with GCC 5
The original ALVAR sources don't compile with GCC 5, the default compiler on Ubuntu 16.04. We forked a mirror of ALVAR and fixed it; the fork is at github.com/meetrix/alvar.
git clone https://github.com/meetrix/alvar.git
cd alvar/build
chmod +x generate*.sh
./generate_gcc5.sh
cd build_gcc5_release
make
sudo make installmake install puts the ALVAR headers and libraries where a Kurento module can link against them.
Using ALVAR inside a Kurento module
Kurento modules are C++ plugins for the media server. You describe the module's interface in a .kmd.json file, and Kurento's code generator creates the C++ skeleton and the client API from it. Our module was called MeetrixRemoteSupportAlpha, so the interface file lived at:
src/server/interface/meetrixremotesupportalpha.MeetrixRemoteSupportAlpha.kmd.jsonThe augmented reality processing goes in the OpenCV implementation class, MeetrixRemoteSupportAlphaOpenCVImpl.hpp and its .cpp. Kurento hands that class each video frame as an OpenCV matrix. In it, the module:
- passes the frame to ALVAR's marker detector,
- takes the pose of each detected marker,
- renders the 3D model at that pose with Irrlicht, and
- draws the rendered model back onto the frame, which Kurento then sends on to the WebRTC viewers.
Add ALVAR and Irrlicht to the module's CMakeLists.txt so the build links against them, then build and install the module the same way as any other Kurento filter. The OpenCV filter guide linked above walks through that module build step by step, and a TURN server is worth setting up before you test with real users, because camera streams from mobile networks rarely connect without one.
If you are choosing a media server for new work rather than reviving Kurento, compare the options in the best open source WebRTC media servers.
