Just a little hint for myself:

cmake -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$PWD/install \
-DWITH_CUDA=ON -DWITH_OPENMP=ON \
-DOPENCV_EXTRA_MODULES_PATH=<path to opencv contrib repo>/modules \
-DOPENCV_CUDA_FORCE_BUILTIN_CMAKE_MODULE=ON \
-DWITH_PROTOBUF=ON \
-DOPENCV_EXTRA_SHARED_LINKER_FLAGS="-lomp" \
<path to opencv source>

Important things here:

• OPENCV_CUDA_FORCE_BUILTIN_CMAKE_MODULE – is required, due to some weird staff in cmake (cmake internal FindCUDA.cmake seems to be wrong), this comment on opencv github by alalek was extemely usefull and provided us with right hint.
• WITH_PROTOBUF is also required, otherwise it fails with crazy libprotobuf linker errors.
• OPENCV_EXTRA_SHARED_LINKER_FLAGS=”-lomp” If you’re using clang 10 or older, you’ll need to to add “-lomp” to linker flags manually, because it ignores “-fopenmp” flag which cmake passes to clang driver in this case. Look’s like it falls just here:
  

• Rest of flags looks pretty obvious