ROS 1 Noetic 视觉应用

本章介绍了 reComputer Jetson 上的实用 ROS 1 视觉工作流程，包括相机预览、QR 识别、姿态估计、对象检测、AR 标记、OpenCV 和 MediaPipe 示例。

长时间运行的示例保存在./code/下，相关图形保存在./images/下。

内容

7.2.2.1 首先阅读本文

一切都与使用有关。

ROS 1环境位于Docker镜像中，需要Docker基础才能完全运行！

注意：Docker 镜像只能运行使用 USB 摄像头的视觉案例！

进入 Docker。

打开终端在ROS 1的docker镜像中输入以下命令

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

数字

7.2.2.1 首先阅读 图 1

7.2.2.2 相机预览

相机预览

进入ROS 1 Docker镜像

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

查看相机支持的参数

输入以下命令查看USB摄像头映射的设备名称：

bash
ls /dev/video*

根据对应的设备索引号（0到n）查看USB摄像头支持的流格式、帧数、分辨率：

bash
# Install v4l2 utilities if they are not already installed:
apt update
apt install v4l-utils ffmpeg
bash
v4l2-ctl -d /dev/video0 --list-formats-ext

相机预览图像 -- 终端命令

终端可以用ffplay打开USB摄像头

bash
ffplay -f v4l2 -i /dev/video0

相机预览 - Python 脚本

运行本地USB摄像头来测试脚本：

bash
 cd source_code/ROS 1/
 python3 CameraPreview.py

脚本将通过可用的 USB 摄像头来显示和打印帧速率

CameraPreview.py的源码如下：

代码文件：7-2-2-2-camera-preview-CameraPreview.py

python
#!/usr/bin/env python3
import cv2
import glob
import time
import os

def list_video_devices():
    """Return the /dev/video* device list."""
    devs = sorted(glob.glob("/dev/video*"))
    return devs

def try_open_camera(dev):
    """Try to open a camera device."""
    cap = cv2.VideoCapture(dev, cv2.CAP_V4L2)
    if not cap.isOpened():
        cap.release()
        return None
    return cap

def main():
    devices = list_video_devices()
    if not devices:
        print("❌ No /dev/video* found")
        return

    print("🔍 Found video devices:")
    for d in devices:
        print("  ", d)

    cams = []
    for dev in devices:
        cap = try_open_camera(dev)
        if cap:
            print(f"✅ Opened {dev}")
            cams.append((dev, cap))
        else:
            print(f"❌ Failed {dev}")

    if not cams:
        print("❌ No usable cameras")
        return

    cam_index = 0
    last_time = time.time()
    fps = 0.0

    print("\nControls:")
    print("  n: next camera")
    print("  p: previous camera")
    print("  q / ESC: quit\n")

    while True:
        dev, cap = cams[cam_index]

        ret, frame = cap.read()
        if not ret:
            cv2.putText(
                frame,
                "Camera read failed",
                (30, 50),
                cv2.FONT_HERSHEY_SIMPLEX,
                1,
                (0, 0, 255),
                2,
            )
        else:
            now = time.time()
            fps = 1.0 / (now - last_time)
            last_time = now

            # FPS display
            cv2.putText(
                frame,
                f"FPS: {fps:.2f}",
                (20, 40),
                cv2.FONT_HERSHEY_SIMPLEX,
                1,
                (0, 255, 0),
                2,
            )

            # Device name display
            cv2.putText(
                frame,
                dev,
                (20, frame.shape[0] - 20),
                cv2.FONT_HERSHEY_SIMPLEX,
                0.7,
                (255, 255, 255),
                2,
            )

        cv2.imshow("CameraPreview", frame)

        key = cv2.waitKey(1) & 0xFF
        if key in [27, ord("q")]:  # ESC / q
            break
        elif key == ord("n"):
            cam_index = (cam_index + 1) % len(cams)
            print(f"➡ Switch to {cams[cam_index][0]}")
            time.sleep(0.2)
        elif key == ord("p"):
            cam_index = (cam_index - 1) % len(cams)
            print(f"⬅ Switch to {cams[cam_index][0]}")
            time.sleep(0.2)

    for _, cap in cams:
        cap.release()
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

数字

7.2.2.2 相机预览图1

7.2.2.2 相机预览图2

7.2.2.2 相机预览图3

7.2.2.2 相机预览图4

7.2.2.2 相机预览图5

7.2.2.2 相机预览图6

7.2.2.3 二维码

1个二维码简介

QR码是二维条码的一种，QR来源于英文“Quick Response”的缩写，意为快速反应，来源于发明者对QR码能够快速解码的希望。二维码不仅信息容量大、可靠性高、成本高，而且表明汉字、图像等多种文本信息具有保密性、虚假性，易于被利用。更重要的是，二维码技术是开源的。

二维码简介

###二维码的特点

QR 码中的数据值包含重复信息（冗余值）。这样，即使高达30%的二维码结构被破坏，也不会影响二维码的可读性。 QR码的存储空间多达7089位或4296个字符，包括平底符号和特殊字符，全部写入QR码中。除了数字和字符之外，还可以对单词和短语（例如网站）进行编码。随着更多数据添加到QR码中并且代码尺寸增加，代码结构变得更加复杂。

QR 二维码创建和识别

安装依赖：

代码文件：7-2-2-3-qr-codes-install-deps.sh

bash
sudo apt update
sudo apt install -y python3-pip libzbar-dev
python3 -m pip install qrcode pyzbar

创建一个qrcode对象，可以直接运行该目录下的qr code create.py脚本：

bash
python3 qrcode_create.py

工作目录下的图形文件夹中会生成带有设置徽标的二维码（QR 码）。预设标志图片的位置也在figure文件夹中。生成的二维码效果如下：

qrcode create.py的源码如下：

代码文件：7-2-2-3-qr-codes-create.py

python
#!/usr/bin/env python3
import os
import qrcode
from PIL import Image
from pathlib import Path


def add_logo(img, logo_path):
    # Add logo, open logo image
    icon = Image.open(    logo_path)
    img_w, img_h = img.size
    # Set the size of the logo
    factor = 6
    size_w = int(img_w / factor)
    size_h = int(img_h / factor)
    icon_w, icon_h = icon.size
    if icon_w > size_w: icon_w = size_w
    if icon_h > size_h: icon_h = size_h
    # Resize the logo
    icon = icon.resize((icon_w, icon_h), Image.Resampling.LANCZOS)
    # Center the logo
    w = int((img_w - icon_w) / 2)
    h = int((img_h - icon_h) / 2)
    # Paste the logo
    img.paste(icon, (w, h), mask=None)
    return img


def create_qrcode(data, file_name, logo_path):
    '''
    version: Integer from 1 to 40, controls the size of the QR code.
    error_correction: Controls the error correction function. Can be one of the following:
        ERROR_CORRECT_L: About 7% or fewer errors can be corrected.
        ERROR_CORRECT_M (default): About 15% or fewer errors can be corrected.
        ERROR_CORRECT_H: About 30% or fewer errors can be corrected.
    box_size: Controls the number of pixels in each box of the QR code.
    border: Controls the number of boxes for the border (the default is 4).
    '''
    qr = qrcode.QRCode(
        version=1,
        error_correction=qrcode.constants.ERROR_CORRECT_H,
        box_size=10,
        border=4)
    # Add data to the QR code
    qr.add_data(data)
    print(data)
    qr.make(fit=True)
    img = qr.make_image(fill_color="green", back_color="white")
    # Add logo if file exists
    if Path(logo_path).is_file():
        img = add_logo(img, logo_path)
    # Save and show the image
    img.save(file_name)
    img.show()
    return img


if __name__ == '__main__':
    file_path = "./figures"
    logo_path = file_path + "/seeed_logo.png"
    out_img = file_path + '/seeed_logo_qr.jpg'
    text = input("Please enter:")
    create_qrcode(text, out_img, logo_path)

创建二维码的参数含义如下：

代码文件：7-2-2-3-qr-codes-example-03.py

python
'''
    Parameters:
    version：integer from 1 to 40 that controls QR-code size.
             use None with fit if you want the library to choose automatically.
    error_correction：controls QR-code error correction.
    ERROR_CORRECT_L：about 7% or fewer errors can be corrected.
    ERROR_CORRECT_M (default): about 15% or fewer errors can be corrected.
    ROR_CORRECT_H：about 30% or fewer errors can be corrected.
    box_size：controls the pixel size of each QR-code cell.
    border：controls the border width in cells; the default 4 is the standard minimum.
    '''

生成二维码后，可以将其打印出来（A4大小即可）进行二维识别测试。

识别的功能代码段如下：

代码文件：7-2-2-3-qr-codes-example-04.py

python
def decodeDisplay(image, font_path):
    gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
    # Convert decoded text to Unicode before drawing it on the image.
    barcodes = pyzbar.decode(gray)
    for barcode in barcodes:
        # Extract the QR code bounding box.
        (x, y, w, h) = barcode.rect
        # Draw the barcode bounding box on the image.
        cv.rectangle(image, (x, y), (x + w, y + h), (225, 0, 0), 5)
        encoding = 'UTF-8'
        # Convert the decoded bytes to a string before drawing.
        barcodeData = barcode.data.decode(encoding)
        barcodeType = barcode.type
        # Draw the decoded data and type on the image.
        pilimg = Image.fromarray(image)
        # Create a drawing object.
        draw = ImageDraw.Draw(pilimg)
        # Parameter 1: font file path. Parameter 2: font size.
        fontStyle = ImageFont.truetype(font_path, size=12, encoding=encoding)
        # Parameter 1: text position. Parameter 2: text. Parameter 3: text color. Parameter 4: font.
        draw.text((x, y - 25), str(barcode.data, encoding), fill=(255, 0, 0), font=fontStyle)
        # Convert the PIL image back to an OpenCV image.
        image = cv.cvtColor(np.array(pilimg), cv.COLOR_RGB2BGR)
        # Print barcode data and type to the terminal.
        print("[INFO] Found {} barcode: {}".format(barcodeType, barcodeData))
    return image

在二维码识别目录下，发起二维码的检测：

python
qrcode_parsing_usb.py

测试结果如下图所示，识别二维码后绘制测试框。

数字

7.2.2.3 QR 码图 1

7.2.2.3 QR 码图 2

7.2.2.3 QR 码图 3

7.2.2.4 人体姿态估计

人体姿势估计

简介

人体姿势评估是一种基于计算机的视觉技术，旨在通过图像或视频自动识别和定位人体关键关节的位置，并对人体骨骼结构进行建模。人体姿态估计广泛应用于动作识别、智能监控、人机交互、运动分析、康复和虚拟现实等领域，是连接“看到人体”和“理解人类行为”的关键技术。

二.基本原理

首先，输入的人体图像被发送到体积神经网络（CNN）进行表征，并具有高级语义特征；然后将网络结构分为两个分支，一个用于预测部位置信图（Keypoint Confidence Chart），用于显示单个人体关节在空间中位置的概率分布，另一个用于预测部位亲和力场（PAF，Body Associated Fields），用于以向量形式描述不同关节之间的链接和方向信息。基于检测候选节点的置信图，利用PAF提供的方向一致性信息，将节点之间的连接建模为图中的二分匹配问题，通过正确匹配属于同一个人的关节节点，逐步构建完整的人体骨骼。进一步，估计多人手势可以被认为是一个多人解析问题，将多点连接转化为图形匹配问题，利用经典的二维匹配方法，如匈牙利算法，找到最佳的关节节点组合，从而实现对多个人体手势的准确稳定的估计。

诉讼程序开始

访问 ROS 1 docker 镜像

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

安装依赖

bash
apt-get update
apt-get install -y libatlas-base-dev libprotobuf-dev protobuf-build
apt-get install python3-pip
pip install \
  https://files.pythonhosted.org/packages/c0/b7/20228024ef7bcfaa9cad2cb2514f3f7cc7a2c2c16715dc65cb01c793693c/mediapipe-0.10.9-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

# Check whether the installation succeeded.
python3 -c "import mediapipe as mp; print(mp.__version__)"

使用图片进行推理

进入代码文件目录，运行代码脚本

bash
cd sources_code/ROS 1
python3 target_pose_img.py

使用视频图像推理

运行代码脚本

bash
undefined

数字

7.2.2.4 人体姿态估计图1

7.2.2.4 人体姿态估计图2

7.2.2.4 人体姿态估计图 3

7.2.2.5 物体检测

物体检测

在深度学习中，目标检测是计算机可视化的核心任务之一，旨在识别和标记图像中所有感兴趣的对象。常见的方法主要分为两大类：

两阶段测试： 表示方法有R-CNN、Fast R-CNN、Fast R-CNN。 Mr. Region Products，然后对每个候选区域进行分类并返回。这些方法精度较高，但速度较慢，适合精度要求较高的场景。

一级测试： 表示方法有YOLO（You Only Look Once）、SSD（Single Shot MultiBox Detector）。对象类别和边界框直接投影在特征图上，无需以适合实时检测的快速速度生成候选区域，但早期精度略低于两个阶段。

本标题将演示如何使用OpenCV的dnn模块来实现目标测试

启动程序

###图片推理

进入代码目录，运行脚本

bash
cd sources_code/ROS 1
python3 target_detect_img.py

逻辑相机

进入代码目录，运行脚本

bash
cd sources_code/ROS 1
python3 target_detect_video.py

7.2.2.6 AR视觉

AR 视觉

增强现实（AR，Augmented Reality）是指利用基于计算机的视觉技术，将虚拟信息或三维模型实时叠加到现实世界的图像或视频中，从而增强用户对环境的感知和交互体验。它通过对现实生活场景的视频捕捉，结合目标检测、特征点跟踪、深度估计等技术，实现虚拟物体与现实世界的空间对齐和实时交互，广泛应用于游乐娱乐、工业设计、教育培训、导航等领域。

输入 ROS 1 Docker 镜像

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

相机已标记。

ROS官方提供了相机标定包，可以轻松标记USB相机：

bash
apt install ros-noetic-camera-calibration
apt install ros-noetic-usb-cam

启动USB摄像头节点

bash
roslaunch usb_cam usb_cam-test.launch

托盘的准备

下载并打印 A4 尺寸的平板电脑

开始标记节点

在ROS 1的docker镜像中打开另一个终端，运行标记的节点

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

  rosrun camera_calibration cameracalibrator.py \
  --size 8x6 --square 0.025 image:=/usb_cam/image_raw camera:=/usb_cam

参数说明：

--size 8x6：内角数（列 x 行）（8x6 = 9x7 网格的 48 个角）

--square 0.025：每平方长，单位米（24mm）

图片主题

相机：=/usb_cam：相机命名空间

标记接口

X：摄像机视图中棋格的左右移动

Y：国际象棋窗格在摄像机视图中上下移动

尺寸：相机视图中板子的前后尺寸

倾斜：国际象棋窗格在相机视图中倾斜

成功启动后，棋盘被放置在图片的中央，改变位置。系统会自我识别，最好是X、Y、Size、Skew下面的线尽可能地用从红色到黄色的数据收集来填充。

点击Calibrate计算相机内部，显示界面可能没有任何反应，但终端仍有输出，等待！

计算成功后，终端显示摄像机参数，点击SAVE保存摄像机参数。

结果将默认存储在/tmp/calibrationdata.tar.gz

按下它并移至标记参数。 yaml文件复制到脚本代码目录

bash
mv tmp/ost.yaml /source_code/ROS 1/sources

运行脚本

在 ROS 1 Docker 容器中运行 AR 脚本：

bash
python3 simple_AR.py

代码文件：7-2-2-6-ar-vision-example-01.py

python
#!/usr/bin/env python3
# simple_AR.py
# common lib
import os
import sys
import time
import cv2 as cv
import numpy as np
import yaml

print("import done")

cv_edition = cv.__version__
print("cv_edition: ",cv_edition)

def load_camera_from_ost(yaml_path: str):
    with open(yaml_path, 'r') as f:
        params = yaml.safe_load(f)
    camera_data = params['camera_matrix']['data']
    dist_data = params['distortion_coefficients']['data']
    camera_matrix = np.array(camera_data, dtype=np.float32).reshape(3, 3)
    dist_coeffs = np.array(dist_data, dtype=np.float32).reshape(-1, 1)
    return camera_matrix, dist_coeffs

def draw_stickman(img, img_pts):
    cv.line(img, tuple(img_pts[18]), tuple(img_pts[4]), (0, 0, 255), 3)
    cv.line(img, tuple(img_pts[18]), tuple(img_pts[6]), (0, 0, 255), 3)
    cv.line(img, tuple(img_pts[18]), tuple(img_pts[21]), (0, 0, 255), 3)
    cv.line(img, tuple(img_pts[21]), tuple(img_pts[19]), (0, 0, 255), 3)
    cv.line(img, tuple(img_pts[21]), tuple(img_pts[20]), (0, 0, 255), 3)
    cv.line(img, tuple(img_pts[21]), tuple(img_pts[22]), (0, 0, 255), 3)
    cv.circle(img, tuple(img_pts[22]), 15, (0, 0, 255), -1)
    cv.line(img, tuple(img_pts[74]), tuple(img_pts[72]), (0, 255, 0), 3)
    cv.line(img, tuple(img_pts[74]), tuple(img_pts[73]), (0, 255, 0), 3)
    cv.line(img, tuple(img_pts[74]), tuple(img_pts[37]), (0, 255, 0), 3)
    cv.line(img, tuple(img_pts[37]), tuple(img_pts[76]), (0, 255, 0), 3)
    cv.line(img, tuple(img_pts[37]), tuple(img_pts[77]), (0, 255, 0), 3)
    cv.line(img, tuple(img_pts[37]), tuple(img_pts[75]), (0, 255, 0), 3)
    cv.circle(img, tuple(img_pts[75]), 15, (0, 255, 0), -1)
    return img

def main():
    print("start")

    pattern_size = (8,6)
    yaml_path = os.path.join(os.path.dirname(__file__), 'sources', 'ost.yaml')
    camera_matrix, dist_coeffs = load_camera_from_ost(yaml_path)

    object_points = np.zeros((pattern_size[0] * pattern_size[1], 3), np.float32)
    object_points[:,:2] = np.mgrid[0:pattern_size[0], 0:pattern_size[1]].T.reshape(-1, 2)

    axis = np.float32([
            [0, 0, -1], [0, 8, -1], [5, 8, -1], [5, 0, -1],
            [1, 2, -1], [1, 6, -1], [4, 2, -1], [4, 6, -1],
            [1, 0, -4], [1, 8, -4], [4, 0, -4], [4, 8, -4],
            [1, 2, -4], [1, 6, -4], [4, 2, -4], [4, 6, -4],
            [0, 1, -4], [3, 2, -1], [2, 2, -3], [3, 2, -3],
            [1, 2, -3], [2, 2, -4], [2, 2, -5], [0, 4, -4],
            [2, 3, -4], [1, 3, -4], [4, 3, -5], [4, 5, -5],
            [1, 2, -3], [1, 6, -3], [5, 2, -3], [5, 6, -3],
            [3, 4, -5], [0, 6, -4], [5, 6, -4], [2, 8, -4],
            [3, 8, -4], [2, 6, -4], [2, 0, -4], [1, 5, -4],
            [3, 0, -4], [3, 2, -4], [0, 3, -4], [1, 2, -4],
            [4, 2, -4], [5, 3, -4], [2, 7, -4], [3, 7, -4],
            [3, 3, -1], [3, 5, -1], [1, 5, -1], [1, 3, -1],
            [3, 3, -3], [3, 5, -3], [1, 5, -3], [1, 3, -3],
            [1, 3, -6], [1, 5, -6], [3, 3, -4], [3, 5, -4],
            [0, 0, -4], [3, 1, -4], [1, 1, -4], [0, 2, -4],
            [2, 4, -4], [4, 4, -4], [0, 8, -4], [5, 8, -4],
            [5, 0, -4], [0, 4, -5], [5, 4, -4], [5, 4, -5],
            [2, 5, -1], [2, 7, -1], [2, 6, -3], [2, 6, -5],
            [2, 5, -3], [2, 7, -3]
            ])

    capture = cv.VideoCapture(0)
    if cv_edition[0] == '3':
        capture.set(cv.CAP_PROP_FOURCC, cv.VideoWriter_fourcc(*'XVID'))
    else:
        capture.set(cv.CAP_PROP_FOURCC, cv.VideoWriter.fourcc('M', 'J', 'P', 'G'))
    capture.set(6, cv.VideoWriter_fourcc('M', 'J', 'P', 'G'))
    capture.set(cv.CAP_PROP_FRAME_WIDTH, 640)
    capture.set(cv.CAP_PROP_FRAME_HEIGHT, 480)

    last_time = time.time()
    fps = 0.0

    while True:
        ret, frame = capture.read()
        if not ret:
            break

        now = time.time()
        fps = 1.0 / max(1e-6, (now - last_time))
        last_time = now
        gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)

        retval, corners = cv.findChessboardCorners(
            gray,
            pattern_size,
            None,
            flags=cv.CALIB_CB_ADAPTIVE_THRESH + cv.CALIB_CB_NORMALIZE_IMAGE + cv.CALIB_CB_FAST_CHECK,
        )

        if retval:
            corners = cv.cornerSubPix(
                gray,
                corners,
                (11, 11),
                (-1, -1),
                (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001),
            )
            retval, rvec, tvec, _inliers = cv.solvePnPRansac(
                object_points,
                corners,
                camera_matrix,
                dist_coeffs,
            )
            if retval:
                image_points, _jacobian = cv.projectPoints(
                    axis,
                    rvec,
                    tvec,
                    camera_matrix,
                    dist_coeffs,
                )
                img_pts = np.int32(image_points).reshape(-1, 2)
                frame = draw_stickman(frame, img_pts)

                cv.putText(frame, "AR Active - 2 Stickman", (10, 60), cv.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 0), 2)
            else:
                cv.putText(frame, "Pose estimation failed", (10, 60), cv.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)
        else:
            cv.putText(frame, "No chessboard detected", (10, 60), cv.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)

        cv.putText(frame, f"FPS: {fps:.1f}", (10, 30), cv.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
        cv.imshow('frame', frame)

        action = cv.waitKey(1) & 0xFF
        if action == ord('q') or action == 113:
            break

    capture.release()
    cv.destroyAllWindows()

if __name__ == "__main__":
    main()

数字

7.2.2.6 AR视觉图1

7.2.2.6 AR 视觉图 2

7.2.2.6 AR 视觉图 3

7.2.2.6 AR 视觉图 4

7.2.2.6 AR 视觉图 5

7.2.2.6 AR 视觉图 6

7.2.2.6 AR 视觉图 7

7.2.2.7 OpenCV 应用

OpenCV 应用

Opencv apps 是 ROS 社区维护的一组功能包，它将大量 OpenCV 图像处理和计算机可视化封装到 ROS 节点中，允许标准主题的输入直接在 ROS 系统中调用，而无需自己准备 OpenCV 代码。

启动 Opencv 应用程序

启动docker

bash
sudo docker run -it \
  --name ros_noetic \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

＃＃＃ 安装

bash
apt install ros-noetic-opencv-apps
apt install -y ros-noetic-usb-cam


# Check whether the installation succeeded.
roscd opencv_apps
# Open another terminal.
sudo docker exec -it ros_noetic bash

rosrun opencv_apps face_detection image:=/usb_cam/image_raw

激活相机

bash
roslaunch usb_cam usb_cam-test.launch

使用 Opencv 应用程序

一次只能运行一个函数。激活相机后需要运行命令：

bash
roslaunch opencv_apps face_recognition.launch image:=/usb_cam/image_raw          # Face recognition
roslaunch opencv_apps corner_harris.launch image:=/usb_cam/image_raw             # Harris corner detection
roslaunch opencv_apps camshift.launch image:=/usb_cam/image_raw                  # Object tracking
roslaunch opencv_apps contour_moments.launch image:=/usb_cam/image_raw           # Contour moments
roslaunch opencv_apps convex_hull.launch image:=/usb_cam/image_raw               # Convex hull
roslaunch opencv_apps discrete_fourier_transform.launch image:=/usb_cam/image_raw # Discrete Fourier transform
roslaunch opencv_apps edge_detection.launch image:=/usb_cam/image_raw            # Edge detection
roslaunch opencv_apps face_detection.launch image:=/usb_cam/image_raw            # Face detection
roslaunch opencv_apps fback_flow.launch image:=/usb_cam/image_raw                # Optical flow detection
roslaunch opencv_apps find_contours.launch image:=/usb_cam/image_raw             # Contour detection
roslaunch opencv_apps general_contours.launch image:=/usb_cam/image_raw          # General contour detection
roslaunch opencv_apps goodfeature_track.launch image:=/usb_cam/image_raw         # Feature point tracking
roslaunch opencv_apps hls_color_filter.launch image:=/usb_cam/image_raw          # HLS color filtering
roslaunch opencv_apps hough_circles.launch image:=/usb_cam/image_raw             # Hough circle detection
roslaunch opencv_apps hough_lines.launch image:=/usb_cam/image_raw               # Hough line detection
roslaunch opencv_apps hsv_color_filter.launch image:=/usb_cam/image_raw          # HSV color filtering
roslaunch opencv_apps lk_flow.launch image:=/usb_cam/image_raw                   # LKoptical flow algorithm
roslaunch opencv_apps people_detect.launch image:=/usb_cam/image_raw             # People detection
roslaunch opencv_apps phase_corr.launch image:=/usb_cam/image_raw                # Phase correlation displacement detection
roslaunch opencv_apps pyramids.launch image:=/usb_cam/image_raw                  # Image pyramid
roslaunch opencv_apps rgb_color_filter.launch image:=/usb_cam/image_raw          # RGB color filtering
roslaunch opencv_apps segment_objects.launch image:=/usb_cam/image_raw           # Foreground segmentation
roslaunch opencv_apps simple_flow.launch image:=/usb_cam/image_raw               # Simple optical flow
roslaunch opencv_apps smoothing.launch image:=/usb_cam/image_raw                 # Smoothing filter
roslaunch opencv_apps threshold.launch image:=/usb_cam/image_raw                 # Thresholding
roslaunch opencv_apps watershed_segmentation.launch image:=/usb_cam/image_raw    # Watershed segmentation

预览

激活相机并激活 Opencv 应用程序功能之一，以通过以下方式查看图像。

本地访问

输入以下命令，选择对应主题即可查看效果：

rqt 图像视图

局域网观看

同一局域网下，在浏览器中输入IP:端口（8080），例如：

192.168.7.47:8080 # IP 指主机IP

参考文献

维基：http://wiki.ros.org/opencv_apps

来源：https://github.com/ros-perception/opencv_apps.git

大部分代码最初取自https://github.com/Itseez/opencv/tree/master/samples/cpp.

7.2.2.8 ROS + OpenCV 基础知识

功能概述

ROS在安装时已经组装了OpenCV（3.x及以上版本），图像在RS中以sensor msgs/Image消息格式传输，无法直接用OpenCV处理。为此，ROS 提供了 cv 桥：

ROS Image ↔ OpenCV Mat（nompy array）之间的高效转换是 ROS 图像处理的中心桥梁。

功能包依赖配置

包.xml

代码文件：7-2-2-8-ros-plus-opencv-fundamentals-package.xml

bash
<?xml version="1.0"?>
<package format="2">
  <name>ros_opencv_demo</name>
  <version>0.0.1</version>
  <description>
    ROS + OpenCV sample package that uses cv_bridge to subscribe to USB camera images and process them with OpenCV.
  </description>

  <!-- maintainer information -->
  <maintainer email="you@example.com">your_name</maintainer>

  <!-- open-source license -->
  <license>BSD</license>

  <!-- build tool -->
  <buildtool_depend>catkin</buildtool_depend>

  <!-- ================= core dependencies ================= -->

  <!-- Python / ROS -->
  <depend>rospy</depend>

  <!-- ROS messages -->
  <depend>std_msgs</depend>
  <depend>sensor_msgs</depend>

  <!-- image-related dependencies -->
  <depend>cv_bridge</depend>
  <depend>image_transport</depend>

  <!-- Optional: enable this if you use PCL later. -->
  <!-- <depend>pcl_ros</depend> -->

  <!-- ================= optional exports ================= -->
  <export>
  </export>

</package>

CMakeLists.txt（Python 节点最小化）

代码文件：7-2-2-8-ros-plus-opencv-fundamentals-example-02.cmake

bash
cmake_minimum_required(VERSION 3.0.2)
project(ros_opencv_demo)
find_package(catkin REQUIRED COMPONENTS
  roscpp
  rospy
  std_msgs
  sensor_msgs
  cv_bridge
  image_transport
)
catkin_package()
include_directories(
  ${catkin_INCLUDE_DIRS}
)

启动USB摄像头节点

bash
# Install dependencies if needed.
apt install -y ros-noetic-usb-cam
roslaunch usb_cam usb_cam-test.launch

彩色图像订阅节点（完整代码）

文件名：usb_cam_image.py

代码文件：7-2-2-8-ros-plus-opencv-fundamentals-usb_cam_image.py

bash
#!/usr/bin/env python3

import rospy
import cv2
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError


class UsbCamImageNode:
    def __init__(self):
        rospy.init_node("usb_cam_image_node", anonymous=True)

        self.bridge = CvBridge()

        # subscribe to USB camera images
        self.image_sub = rospy.Subscriber(
            "/usb_cam/image_raw",
            Image,
            self.image_callback,
            queue_size=1
        )

        rospy.loginfo("USB Camera Image Subscriber Started")

    def image_callback(self, msg):
        try:
            # ROS Image → OpenCV BGR
            frame = self.bridge.imgmsg_to_cv2(msg, desired_encoding="bgr8")
        except CvBridgeError as e:
            rospy.logerr(e)
            return

        # display the image
        cv2.imshow("USB Camera Image", frame)
        cv2.waitKey(1)


if __name__ == "__main__":
    try:
        UsbCamImageNode()
        rospy.spin()
    except rospy.ROSInterruptException:
        pass
    finally:
        cv2.destroyAllWindows()

授予执行权

bash
chmod +x usb_cam_image.py
cd /ros_ws
catkin_make

运行示例

安装多终端工具

bash
apt-get install terminator
# Run
terminator

在弹出的终端界面中使用 ctrl + Shift + o 快捷键水平打开另一个终端

终端 1：启动相机。

Roslaunch usb cam usb cam-test.launch

终端 2：运行订阅节点

rosrun ros opencv 演示 usb_cam_image.py

您将看到一个实时显示 USB 摄像头的 OpenCV 窗口。

查看节点通信

启动另一个终端操作：

bash
rqt_graph

数字

7.2.2.8 ROS + OpenCV 基础知识图 1

7.2.2.8 ROS + OpenCV 基础知识图 2

7.2.2.8 ROS + OpenCV 基础知识图 3

7.2.2.9 MediaPipe 开发

MediaPipe 简介

MediaPipe是Google开源的高性能跨平台传感器计算框架，致力于构建实时可视化和多模块处理电流线路。它通过图形计算（Graph）将摄像头输入、模型推理和再处理模块有效地链接起来，广泛用于人体姿态估计、手部识别、人脸检测和关键点跟踪等任务。 MediaPipe融合了多种轻定量深度学习模型，支持CPU实时推理，对嵌入式平台（如Jetson、移动）非常友好，开发者可以用少量代码快速实现稳定、低延迟的实时感知应用。

MediaPipe 中的深度学习解决方案

单击图片查看完整的电子表格

启动docker

bash
sudo docker run -it \
  --net=host \
  --privileged \
  -v /dev:/dev \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  ros:noetic

使用案例

手测

bash
cd sources_code/ROS 1/mediapipe
python3 hand.py

面部检查。

bash
cd sources_code/ROS 1/mediapipe
python3 face_detection.py

脸部效果。

bash
cd sources_code/ROS 1/mediapipe
python3 face_effect.py

D 对象识别

bash
cd sources_code/ROS 1/mediapipe
python3 object_reg.py

＃＃ 刷子

bash
cd sources_code/ROS 1/mediapipe
python3 virtual_brush.py

齿轮识别。

bash
cd sources_code/ROS 1/mediapipe
python3 gesture_recognition.py

手指控制。

bash
cd sources_code/ROS 1/mediapipe
python3 hand_control.py

装腔作势

bash
cd sources_code/ROS 1/mediapipe
python3 pose_estimation.py

人脸测试

bash
cd sources_code/ROS 1/mediapipe
python3 face_mesh.py

整体测试

bash
cd sources_code/ROS 1/mediapipe
python3 holistic.py

案例分割

bash
cd sources_code/ROS 1/mediapipe
python3 selfie_segmentation.py

数字

7.2.2.9 MediaPipe 开发图1

7.2.2.9 MediaPipe 开发图2

7.2.2.9 MediaPipe 开发图 3

7.2.2.9 MediaPipe 开发图 4

7.2.2.9 MediaPipe 开发图 5

7.2.2.9 MediaPipe 开发图 6

7.2.2.9 MediaPipe 开发图 7

7.2.2.9 MediaPipe 开发图 8

7.2.2.9 MediaPipe 开发图 9

7.2.2.9 MediaPipe 开发图 10

7.2.2.9 MediaPipe 开发图 11

7.2.2.9 MediaPipe 开发图 12

7.2.2.9 MediaPipe 开发图 13