ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
1

OpenCV imshow toolbar missing in noetic

asked 2021-12-02 17:24:44 -0500

Jay Gowdy gravatar image

I've been using OpenCV with ROS kinetic for years, and have come to rely on the functions toolbar that comes with cv::imshow for debugging.

We've successfully transitioned to noetic under Ubuntu 20.04, but now the toolbars in my OpenCV cv::imshow windows are gone, and I can't pan, zoom, or inspect pixel values. The images themselves show up just fine. I just can't do anything with them anymore.

Digging in,

  • it appears that I only get the toolbar with OpenCV installations that are built with the WITH_QT=ON CMake option. Is this correct? Is there another way to get the toolbar/pan/zoom/inspect functionality that I am missing?
  • Noetic appears to use a standard OpenCV 4.2 package installed as a ubuntu 20.04 package instead of using the built-in-to-ROS version of OpenCV 3.3.1 that kinetic under 16.04 used
  • I guess the standard Ubuntu 20.04 OpenCV 4.2 package is built without WITH_QT=ON?

Do I really have strip out the standard openCV from my installation and build my own version of OpenCV from source to get back the functionality I've had for the last 5 years out of the box with kinetic? Is there some other path that I am missing?

If I do have to build my own version of OpenCV from source to get the toolbars, any advice on how to do that for a Docker overlay without rebuilding OpenCV every time? Do I have to build my own .deb and install that as part of the docker overlay creation process, or does a version of OpenCV 4.2 that does what I want exist out there in a repository somewhere? Perhaps I just have to rebuild/load the individual libopencv-highgui4.2 package instead of the whole openCV system? Any advice or pointers on exactly how to do this before I just start flailing at things I only partially understand would be most helpful.

edit retag flag offensive close merge delete

Comments

Note, I did try to explicitly create a cv::namedWindow with the WINDOW_AUTOSIZE | WINDOW_KEEPRATIO | WINDOW_GUI_EXPANDED flags. No dice. The documentation for namedWindow on https://docs.opencv.org/3.4/d7/dfc/gr... does confirm that with the Qt backend these should be the defaults anyway, so more evidence that the default noetic openCV highgui debian package is not built with the Qt backend enabled.

Jay Gowdy gravatar image Jay Gowdy  ( 2021-12-02 21:34:03 -0500 )edit

1 Answer

Sort by » oldest newest most voted
1

answered 2021-12-03 16:03:23 -0500

Jay Gowdy gravatar image

Well, flailing seemed to get me where I needed to go, but it is not pretty. I would really appreciate someone telling me there was a better way.

First, you must get the source tarball for OpenCV 4.2.0 from the OpenCV website, https://github.com/opencv/opencv/arch..., unzip it and you will have a directory opencv-4.2.0

Then, in the same directory above opencv-4.2.0 (for the purposes of these instructions), get the source of the opencv debian package,

apt-get source libopencv-dev

which will create a directory opencv-4.2.0+dfsg

Now, I found that the source directory from the package is missing some critical files for enabling the Qt back end, so copy them over from the original OpenCV source (this is why you have to download it as well):

cp -a opencv-4.2.0/modules/highgui/src/files_Qt/Milky opencv-4.2.0+dfsg/modules/highgui/src/files_Qt

Now, cd to opencv-4.2.0+dfsg and we can forget about the OpenCV source. Edit debian/rules, and change the line that says

-DWITH_QT=OFF \

to be

-DWITH_QT=ON \

We need to make sure a new support library that is created as a result gets installed with highgui, so edit debian/libopencv-highgui-dev.install and add the lines

usr/include/opencv4/opencv2/cvv/*
usr/include/opencv4/opencv2/cvv.hpp
usr/lib/*/libopencv_cvv.a
usr/lib/*/libopencv_cvv.so

and then edit the file debian/libopencv-highgui4.2.install and add the line

usr/lib/*/libopencv_cvv.so.*

Then you have to get setup to build and actually build the .deb files

sudo apt-get update
sudo apt-get install dpkg-dev devscripts
sudo apt build-dep opencv
sudo apt-get install qt5-default
sudo debuild -b -uc -us

You should have a bunch of .deb files in .., and the two important ones are libopencv-highgui4.2_4.2.0+dfsg-5_amd64.deb and libopencv-highgui-dev_4.2.0+dfsg-5_amd64.deb. I gather those two .debs in a debs subdirectory of my developer dockerenv directory that overlays what I as a developer need on top of the images used for deploying on the robots. I add the following to dockerenv/Dockerfile

 # below is a highly risky way of enabling the Qt backend for OpenCV highgui.
 # If you are not actively debugging image processing code, you may want to
 # comment it out 
 COPY debs /home/acr/debs 
 RUN dpkg -r --force-depends libopencv-highgui-dev libopencv-highgui4.2
 RUN dpkg -i /home/acr/debs/libopencv-highgui4.2_4.2.0+dfsg-5_amd64.deb /home/acr/debs/libopencv-highgui-dev_4.2.0+dfsg-5_amd64.deb

This is a fairly alarming and risky feeling maneuver, and it leaves the apt system in my docker pretty much crippled, as everything that depends on highgui is all borked up.

Even more alarming and an indication of how risky this is, when I run with this I got a core dump in an invocation of pcl::ConvexHull::reconstruct. Googling led me to a fortunate thing: I just changed

 #include <pcl/surface/convex_hull.h>

to

#include <pcl/surface/impl/convex_hull.hpp>

That doesn't make me worried at ... (more)

edit flag offensive delete link more

Comments

Thank you for the thorough answer. I’m sure others will benefit in the future as well

osilva gravatar image osilva  ( 2021-12-03 17:22:28 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2021-12-02 17:24:44 -0500

Seen: 662 times

Last updated: Dec 03 '21