网站伪静态,企业网站管理系统推荐,网站开发 需求,公司做手机网站建设目录 1. 任务要求2. 数据集3. 基于homography的特征匹配算法4. 拼接流程展示4.1 图片实例4.2 特征点位图4.3 特征点匹配结果4.4 相机校准结果4.5 拼接结果 5. 部分图像拼接结果展示 1. 任务要求
输入#xff1a;同一个场景的两张待拼接图像#xff08;有部分场景重合#x… 目录 1. 任务要求2. 数据集3. 基于homography的特征匹配算法4. 拼接流程展示4.1 图片实例4.2 特征点位图4.3 特征点匹配结果4.4 相机校准结果4.5 拼接结果 5. 部分图像拼接结果展示 1. 任务要求
输入同一个场景的两张待拼接图像有部分场景重合。任务从输入的两张图像中提取特征点和描述子可以使用现有的图像处理库来执行此任务。自己实现特征匹配算法将来自两张图像的特征点进行匹配。最后根据匹配的特征点估计单应性变换从而通过映射拼接成一张全景图。输出 显示两张图像上提取的特征点的位置显示特征点匹配对应的结果显示经过几何变换后的图像叠加的结果显示最终拼接的结果。 2. 数据集
其中两组图像“cat”和“bridge”拍摄于杭州。其他图像分别来自测试数据链接和其他来源。 3. 基于homography的特征匹配算法
基于homography的特征匹配算法在图像拼接中起着关键作用它能够定位和匹配两张待拼接图像中的特征点从而实现图像的对齐和融合。该算法主要包括以下实现步骤
特征点提取和描述使用ORB和SIFT等特征检测器对待拼接图像进行特征点提取。这些特征点具有在不同尺度和旋转下的不变性。对每个特征点计算其对应的特征描述子用于后续的特征匹配。特征匹配对两幅待拼接图像中的特征点进行匹配。我们使用基于最近邻的匹配其中对于每个特征点找到其在另一幅图像中的最佳匹配点。通过计算特征描述子之间的距离或相似度确定最佳匹配点。计算homography矩阵使用筛选后的特征点匹配对应的坐标计算homography矩阵。homography矩阵可以将一个图像上的点映射到另一个图像上从而实现图像的对齐。图像校准和拼接使用计算得到的homography矩阵对多张图像进行透视变换使其对齐。将校准后的图像进行融合生成拼接结果图像。
import mathimport cv2 as cv
import numpy as npclass FeatureMatcher:def __init__(self, matcher_typehomography, range_width-1, **kwargs):if matcher_type homography:if range_width -1:self.matcher cv.detail_BestOf2NearestMatcher(**kwargs)else:self.matcher cv.detail_BestOf2NearestRangeMatcher(range_width, **kwargs)else:raise ValueError(Unknown matcher type)def match_features(self, features, *args, **kwargs):pairwise_matches self.matcher.apply2(features, *args, **kwargs)self.matcher.collectGarbage()return pairwise_matchesstaticmethoddef draw_matches_matrix(imgs, features, matches, conf_thresh1, inliersFalse, **kwargs):matches_matrix FeatureMatcher.get_matches_matrix(matches)for idx1, idx2 in FeatureMatcher.get_all_img_combinations(len(imgs)):match matches_matrix[idx1, idx2]if match.confidence conf_thresh:continueif inliers:kwargs[matchesMask] match.getInliers()yield idx1, idx2, FeatureMatcher.draw_matches(imgs[idx1], features[idx1], imgs[idx2], features[idx2], match, **kwargs)staticmethoddef get_confidence_matrix(pairwise_matches):matches_matrix FeatureMatcher.get_matches_matrix(pairwise_matches)match_confs [[m.confidence for m in row] for row in matches_matrix]match_conf_matrix np.array(match_confs)return match_conf_matrix 4. 拼接流程展示
4.1 图片实例
为了演示图像拼接的整体实现流程这里我们选择一组我本人拍摄的玉泉校内的两只猫和周边环境图——“cat”两幅有重叠画面的原图如下图所示。其中下面那只白猫几乎是静止不动的上面的带橘色斑点的白猫在两幅图中的位置有相对移动。
from stitching.images import Images# 1. load
images, low_imgs, medium_imgs, final_imgs load_images(img_path)
images_to_match medium_imgs# 2. plot original images
plot_images(images_to_match, (20, 20), savef{save_path}/1-original.png)# 3. print image size
print(fOriginal image size: {images_to_match[0].shape})################ Load images ####################
def load_images(img_path):images Images.of(img_path)medium_imgs list(images.resize(Images.Resolution.MEDIUM))low_imgs list(images.resize(Images.Resolution.LOW))final_imgs list(images.resize(Images.Resolution.FINAL))return images, low_imgs, medium_imgs, final_imgs################ Plot function####################
def plot_image(img, figsize_in_inches(10, 10), saveNone):
N_image 1def plot_images(imgs, figsize_in_inches(10, 10), saveNone):
N_images 14.2 特征点位图
根据特征检测器提取的特征点生成特征点位置图。这里我们以ORB特征检测器为例下图中的绿色小圈展示了待拼接图像中检测到的特征点的分布情况。
from stitching.feature_detector import FeatureDetector# 4. Feature detection: ORB, SIFT
finder FeatureDetector(detectordetector)
features [finder.detect_features(img) for img in images_to_match]key_points_img []
for i in range(len(images_to_match)):key_points_img.append(finder.draw_keypoints(images_to_match[i], features[i]))plot_images(key_points_img, (20, 20), savef{save_path}/2-key_points.png)4.3 特征点匹配结果
通过homography特征匹配算法具体代码见第3节将两张待拼接图像中匹配的特征点进行连接生成特征点匹配结果图。下图中的绿色线段展示了特征点之间的对应关系。
from Feature_matcher import *# 5. Feature matching: homography
matcher FeatureMatcher()
matches matcher.match_features(features)print(matcher.get_confidence_matrix(matches))# 6. plot matching
all_relevant_matches matcher.draw_matches_matrix(images_to_match, features, matches, conf_thresh1,inliersTrue, matchColor(0, 255, 0))for idx1, idx2, img in all_relevant_matches:print(fMatches Image {idx1 1} to Image {idx2 1})plot_image(img, (20, 10), savef{save_path}/3-matching.png)4.4 相机校准结果
根据homography矩阵对两张图像进行透视变换使其对齐生成校准结果图。下图的子图a为校准过程得到的mask图子图b展示了经过校准后的待拼接图像最终拼接图的大小与待拼接图像的大小一致。
from stitching.camera_estimator import CameraEstimator
from stitching.camera_adjuster import CameraAdjuster
from stitching.camera_wave_corrector import WaveCorrector
from stitching.warper import Warper
from stitching.timelapser import Timelapser# 7. Camera Estimation, Adjustion and Correction
cameras camera_correction(features, matches)# 8. Warp images
(warped_low_imgs, warped_low_masks, low_corners, low_sizes,warped_final_imgs, warped_final_masks, final_corners, final_sizes, frame) \ warp_image(images, cameras, low_imgs, final_imgs)plot_images(warped_low_imgs, (10, 10), savef{save_path}/4-warped_low_imgs.png)
plot_images(warped_low_masks, (10, 10), savef{save_path}/4-warped_low_masks.png)
plot_images(frame, (20, 10), savef{save_path}/4-warped_final_imgs.png)################ Camera Estimation ##################
def camera_correction(features, matches):camera_estimator CameraEstimator()camera_adjuster CameraAdjuster()wave_corrector WaveCorrector()cameras camera_estimator.estimate(features, matches)cameras camera_adjuster.adjust(features, matches, cameras)cameras wave_corrector.correct(cameras)return cameras
################ Warp images ####################
def warp_image(images, cameras, low_imgs, final_imgs):warper Warper()warper.set_scale(cameras)low_sizes images.get_scaled_img_sizes(Images.Resolution.LOW)camera_aspect images.get_ratio(Images.Resolution.MEDIUM,Images.Resolution.LOW) # since cameras were obtained on medium imgswarped_low_imgs list(warper.warp_images(low_imgs, cameras, camera_aspect))warped_low_masks list(warper.create_and_warp_masks(low_sizes, cameras, camera_aspect))low_corners, low_sizes warper.warp_rois(low_sizes, cameras, camera_aspect)final_sizes images.get_scaled_img_sizes(Images.Resolution.FINAL)camera_aspect images.get_ratio(Images.Resolution.MEDIUM, Images.Resolution.FINAL)warped_final_imgs list(warper.warp_images(final_imgs, cameras, camera_aspect))warped_final_masks list(warper.create_and_warp_masks(final_sizes, cameras, camera_aspect))final_corners, final_sizes warper.warp_rois(final_sizes, cameras, camera_aspect)# Timelapsertimelapser Timelapser(as_is)timelapser.initialize(final_corners, final_sizes)frame []for img, corner in zip(warped_final_imgs, final_corners):timelapser.process_frame(img, corner)frame.append(timelapser.get_frame())return (warped_low_imgs, warped_low_masks, low_corners, low_sizes,warped_final_imgs, warped_final_masks, final_corners, final_sizes, frame)4.5 拼接结果
将经过校准的两张图像进行融合生成拼接结果图。根据用户的选择可以提供剪裁相机校准结果的选项stitching(crop True)默认为False。图1分别展示了未剪裁和剪裁后的校准图5ac和拼接图时的接缝5bd。最后拼接图结果见图2上面三幅图不包括剪裁步骤下面三幅存在剪裁步骤。可以看到在拼接之前剪裁至规则的四边形对拼接时的seam line的选取有较大的影响有一定概率导致最终的拼接图像不符合预期。
from stitching.cropper import Cropper
from stitching.seam_finder import SeamFinder# 9. Crop images
if crop:(cropped_low_imgs, cropped_low_masks, cropped_final_imgs,cropped_final_masks, final_corners, final_sizes, frame) (crop_image(images, warped_low_imgs, warped_low_masks, low_corners, low_sizes,warped_final_imgs, warped_final_masks, final_corners, final_sizes))plot_images(frame, (20, 10), savef{save_path}/5-cropped_final_imgs.png)
else:cropped_low_imgs warped_low_imgscropped_low_masks warped_low_maskscropped_final_imgs warped_final_imgscropped_final_masks warped_final_masks# 10. Seam Masks
seam_finder, seam_masks_plots, compensated_imgs, seam_masks (seam(cropped_low_imgs, low_corners, cropped_low_masks,cropped_final_masks, cropped_final_imgs, final_corners))
plot_images(seam_masks_plots, (15, 10), savef{save_path}/6-seam_masks.png)# 11. Matching result
blender Blender()
blender.prepare(final_corners, final_sizes)
for img, mask, corner in zip(compensated_imgs, seam_masks, final_corners):blender.feed(img, mask, corner)
panorama, _ blender.blend()
blended_seam_masks seam_finder.blend_seam_masks(seam_masks, final_corners, final_sizes)plot_image(panorama, (20, 20), savef{save_path}/7-matched_result.png)
plot_image(seam_finder.draw_seam_lines(panorama, blended_seam_masks, linesize3), (15, 10),savef{save_path}/8-seam_lines.png)
plot_image(seam_finder.draw_seam_polygons(panorama, blended_seam_masks), (15, 10),savef{save_path}/9-seam_polygons.png)# 12. Done
print(Done!)################ Crop images ####################
def crop_image(images, warped_low_imgs, warped_low_masks, low_corners, low_sizes,warped_final_imgs, warped_final_masks, final_corners, final_sizes):cropper Cropper()mask cropper.estimate_panorama_mask(warped_low_imgs, warped_low_masks, low_corners, low_sizes)lir cropper.estimate_largest_interior_rectangle(mask)low_corners cropper.get_zero_center_corners(low_corners)rectangles cropper.get_rectangles(low_corners, low_sizes)overlap cropper.get_overlap(rectangles[1], lir)intersection cropper.get_intersection(rectangles[1], overlap)cropper.prepare(warped_low_imgs, warped_low_masks, low_corners, low_sizes)cropped_low_masks list(cropper.crop_images(warped_low_masks))cropped_low_imgs list(cropper.crop_images(warped_low_imgs))low_corners, low_sizes cropper.crop_rois(low_corners, low_sizes)lir_aspect images.get_ratio(Images.Resolution.LOW, Images.Resolution.FINAL) # since lir was obtained on low imgscropped_final_masks list(cropper.crop_images(warped_final_masks, lir_aspect))cropped_final_imgs list(cropper.crop_images(warped_final_imgs, lir_aspect))final_corners, final_sizes cropper.crop_rois(final_corners, final_sizes, lir_aspect)# Redo the timelapse with cropped Images:timelapser Timelapser(as_is)timelapser.initialize(final_corners, final_sizes)frame []for img, corner in zip(cropped_final_imgs, final_corners):timelapser.process_frame(img, corner)frame.append(timelapser.get_frame())return (cropped_low_imgs, cropped_low_masks, cropped_final_imgs,cropped_final_masks, final_corners, final_sizes, frame)图1. Crop images. a, Warpped. b, Wrapped to seam. c, Warp and crop. b, Cropped to seam. 图2. Stitching results. a, Original result. b, Result with seam line. c, Result with seam ploygons. d-f, Cropped results. 5. 部分图像拼接结果展示 Fig. S1. Examples using ORB detector and without cropmask. a, bridge. b, building. c, sportfield. d, door. e, barcode. f, exposure_error. Left, Original. Middle, Stitching results. Right, Result with seam ploygons. 创作不易麻烦点点赞和关注咯
