On this page
Kurento's big differentiator among WebRTC media servers is that it lets you run your own code on the video as it flows. The easiest way in is an OpenCV filter: Kurento hands your code each frame as an OpenCV matrix, you change it, and it goes out to viewers. In this guide the filter writes text onto the video, which sounds trivial but covers every step a real computer vision module needs.
We first published this in 2018 for Kurento 6.7 on Ubuntu 16.04, with a companion sample repository. The code idea is unchanged; the tooling around it has moved on, and the steps below follow Kurento 7.3's module writing documentation and the current sources on Kurento's GitHub.
How Kurento modules work
A Kurento module has two halves. The server side is a C++ library that Kurento loads, containing your media element. The interface is a .kmd.json file describing the element and its methods, from which Kurento generates client code for JavaScript and Java. Your application creates the element in a pipeline, between a WebRtcEndpoint and wherever the video goes next, the same way it would use a built-in filter.
For an OpenCV module, the part you write is one function: process(cv::Mat &mat), called for every frame.
Set up the build machine
Build on Ubuntu 24.04, the same release Kurento 7.3 runs on. Kurento's docs are explicit that mixing systems between building and running a module causes C++ ABI problems. Add the Kurento repository exactly as in installing Kurento Media Server on Ubuntu 24.04, then install the development package:
sudo apt-get update
sudo apt-get install --no-install-recommends --yes kurento-media-server-devThat package brings the headers, CMake helpers and the scaffolding tool. The 2018 version of this guide used a xenial repository line and apt-key; neither applies any more.
Scaffold the module
The scaffold tool generates the module structure. Its arguments are a namespace prefix, the module name in PascalCase, the name in snake_case, and whether it is an OpenCV module:
kurento-module-scaffold Meetrix HelloWorld hello_world trueThe prefix avoids clashes with other modules, and true generates the OpenCV variant. Change into the generated module directory, then run CMake once to generate the implementation files from the interface definition:
cd hello-world/ # use the directory the scaffold created
mkdir build && cd build
cmake ..The implementation files appear under src/server/implementation/objects/. The one to edit ends in OpenCVImpl.cpp.
Write the frame processing
The generated process method throws a "not implemented" exception. Replace its body:
void HelloWorldOpenCVImpl::process (cv::Mat &mat)
{
cv::putText (mat, textToPrint, cv::Point (positionX, positionY),
cv::FONT_HERSHEY_PLAIN, 2, cv::Scalar (255, 255, 255), 2);
}The frame is a normal cv::Mat, so anything OpenCV can do to an image, you can do here: detect faces, blur regions, draw overlays. Whatever you leave in mat when the function returns is what viewers see.
This runs for every frame
process, load models once in the constructor rather than per frame, and measure CPU with several streams before putting heavy vision code in front of users.
Add a method you can call
Static text is not very useful. To let the application change the text and position at runtime, declare a method in the interface file under src/server/interface/ (the file ending in .kmd.json):
"methods": [
{
"name": "setText",
"doc": "Sets the overlay text and its position",
"params": [
{ "name": "text", "doc": "Text to print", "type": "String" },
{ "name": "x", "doc": "Position x", "type": "int" },
{ "name": "y", "doc": "Position y", "type": "int" }
]
}
]Add the state and the setter to the OpenCV implementation class. In the header:
private:
std::string textToPrint = "Hello from Meetrix";
int positionX = 100;
int positionY = 100;
public:
void setText (const std::string &text, int x, int y);In the OpenCVImpl.cpp file:
void HelloWorldOpenCVImpl::setText (const std::string &text, int x, int y)
{
textToPrint = text;
positionX = x;
positionY = y;
}Rerun cmake .. so the element's generated Impl class picks up the new method, then forward the call from it to the OpenCV class, following the pattern of the generated code. Because setText is called from the API thread while process runs on the media thread, a production module should protect those fields with a mutex.
Build, package and install
Build from the build directory:
cmake ..
makeFor anything beyond local testing, build a Debian package from the module's root directory, so the module installs, upgrades and uninstalls cleanly on the media server:
cd ..
dpkg-buildpackage -b -uc -us
sudo dpkg -i ../*.debCheck that Kurento sees it, then restart the server:
kurento-media-server --list
sudo systemctl restart kurento-media-serverThe old guide also documented a -fPIC linker error that needed a manual CMake edit. The generated build files for current Kurento handle that, so only chase it if you see the relocation R_X86_64_32 error yourself.
The libgtk2.0-dev and cvShowImage error
A run that stops with "install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'" means the OpenCV in use was built without a GUI backend. It comes from calling a display function such as cv::imshow inside process. A media server has no screen, so there is nothing to display to. Remove the call and write your changes back into mat, or save a frame with cv::imwrite while debugging. Installing libgtk2.0-dev and rebuilding OpenCV silences the message, but it is the wrong fix on a server.
Use it from JavaScript
Generate the JavaScript client from the same interface file:
cd build
cmake -DGENERATE_JS_CLIENT_PROJECT=TRUE ..The client project appears in build/js. Its package.json contains the module's package name, and lib/index.js shows the exported class name. Install it into your Node.js application, register it with the Kurento client, and create the filter by combining the module name with the class name:
const kurento = require('kurento-client');
kurento.register('kurento-module-helloworld'); // name from build/js/package.json
const webRtcEndpoint = await pipeline.create('WebRtcEndpoint');
const filter = await pipeline.create('helloworld.HelloWorld');
await webRtcEndpoint.connect(filter);
await filter.connect(webRtcEndpoint);
await filter.setText('Hello from Meetrix', 120, 80);A quick way to try it is one of Kurento's Node.js tutorials that already uses a filter, such as the chroma example: swap its filter for yours and remove the chroma-specific calls. When the page loads and you see your text over your camera image, the whole chain from C++ to browser works.
If the filter works locally but not for remote users, the problem is almost always networking rather than the module. Setting up a Coturn TURN server covers what Kurento needs behind NAT, and the open source WebRTC media server comparison is worth a look if you are still deciding whether Kurento's processing model fits your project.
Frequently Asked Questions
What is a Kurento OpenCV filter?
A custom Kurento module that receives every video frame as an OpenCV cv::Mat, lets your C++ code modify it, and passes the result down the media pipeline. It is the simplest way to add computer vision, overlays or image processing to live WebRTC streams in Kurento.
Can I build a Kurento module on a different Ubuntu version than the server?
Don't. Kurento's documentation warns that modules must be built on the same system Kurento was built for, because of C++ ABI compatibility. For Kurento 7.3 that means building on Ubuntu 24.04, the same release your media server runs.
How do I check that Kurento loaded my module?
Run kurento-media-server --list after installing the module package. Your module's name should appear in the list of available modules. If it doesn't, the library is not in Kurento's module path or failed to load, and the server log says why.
Does an OpenCV filter slow down the video?
It can. Your process function runs for every frame, and the frames must be decoded before and re-encoded after the filter. Keep per-frame work small, avoid allocating large buffers on each call, and watch CPU as you add streams.
How do I fix the libgtk2.0-dev error in cvShowImage?
Stop calling cv::imshow from your filter. The message means OpenCV was built without a GUI backend, and a media server has no display anyway. Modify the frame in place, or write it out with cv::imwrite while debugging.
