网站建设的简历范文,电商网站建设教学总结,wordpress登录地址修改密码,江门招聘网最新招聘文章目录 1. Motivation2. Contribution3. Methods3.1 BayerUnify3.2 BayerAug 4. CommentsReference 1. Motivation
对于RAW域去噪#xff0c;通常会将单通道bayer格式的RAW图打包成4通道#xff0c;然后送入神经网络。不同厂家生产的sensor出的RAW图可能具有不同的bayer模… 文章目录 1. Motivation2. Contribution3. Methods3.1 BayerUnify3.2 BayerAug 4. CommentsReference 1. Motivation
对于RAW域去噪通常会将单通道bayer格式的RAW图打包成4通道然后送入神经网络。不同厂家生产的sensor出的RAW图可能具有不同的bayer模式通常是RGGBBGGR GRBG和GBRG。 业内做AI-ISP的攻城狮们应该都会遇到这样一个问题在适配不同sensor的过程中会积累大量具有不同Bayer模式的数据然后在训练模型时都想用上这时大家都会将这些异源的数据统一成相同的bayer模式常用的操作有① 在裁剪patch时根据目标bayer模式选择合适的起点② 打包成4通道然后交换通道顺序。论文作者发现第二种方式会产生伪影而第一种方式不会。 另外数据增强是训练神经网络时提升性能的一种常用手段对于RAW数据为了避免破坏bayer模式通常会选择在打包成4通道后再做翻转和旋转等增强。然而作者发现这样也会产生伪影并提出了相应的解决方案。
2. Contribution
提出了BayerUnify将不同的bayer模式转换为一个统一的模式充分利用异源数据扩大训练集规模提出了BayerAug一种有效的RAW图像的数据增强方式
3. Methods
3.1 BayerUnify
训练阶段采用crop的方式将当前bayer模式转换为目标bayer模式 推理阶段采用先pad的方式转换bayer模式crop会丢失信息对神经网络的输出再做crop得到与原始图像格式一致的结果。
def bayer_unify(raw: np.ndarray, input_pattern: str, target_pattern: str, mode: str) - Tuple:Convert a bayer raw image from one bayer pattern to another.Parameters----------raw : np.ndarray in shape (H, W)Bayer raw image to be unified.input_pattern : {RGGB, BGGR, GRBG, GBRG}The bayer pattern of the input image.target_pattern : {RGGB, BGGR, GRBG, GBRG}The expected output pattern.mode: {crop, pad}The way to handle submosaic shift. crop abandons the outmost pixels,and pad introduces extra pixels. Use crop in training and pad intesting.if input_pattern not in BAYER_PATTERNS:raise ValueError(Unknown input bayer pattern!)if target_pattern not in BAYER_PATTERNS:raise ValueError(Unknown target bayer pattern!)if mode not in NORMALIZATION_MODE:raise ValueError(Unknown normalization mode!)if not isinstance(raw, np.ndarray) or len(raw.shape) ! 2:raise ValueError(raw should be a 2-dimensional numpy.ndarray!)if input_pattern target_pattern:h_offset, w_offset 0, 0elif input_pattern[0] target_pattern[2] and input_pattern[1] target_pattern[3]:h_offset, w_offset 1, 0elif input_pattern[0] target_pattern[1] and input_pattern[2] target_pattern[3]:h_offset, w_offset 0, 1elif input_pattern[0] target_pattern[3] and input_pattern[1] target_pattern[2]:h_offset, w_offset 1, 1else: # This is not happening in [RGGB, BGGR, GRBG, GBRG]raise RuntimeError(Unexpected pair of input and target bayer pattern!)if mode pad:out np.pad(raw, [[h_offset, h_offset], [w_offset, w_offset]], reflect)elif mode crop:h, w raw.shapeout raw[h_offset:h - h_offset, w_offset:w - w_offset]else:raise ValueError(Unknown normalization mode!)return out, h_offset, w_offset3.2 BayerAug
直接对RAW数据做翻转会改变bayer模式BayerAug先翻转再执行BayerUnify保证bayer模式不变。
def bayer_aug(raw: np.ndarray, flip_h: bool, flip_w: bool, transpose: bool, input_pattern: str) - np.ndarray:Apply augmentation to a bayer raw image.Parameters----------raw : np.ndarray in shape (H, W)Bayer raw image to be augmented. H and W must be even numbers.flip_h : boolIf True, do vertical flip.flip_w : boolIf True, do horizontal flip.transpose : boolIf True, do transpose.input_pattern : {RGGB, BGGR, GRBG, GBRG}The bayer pattern of the input image.if input_pattern not in BAYER_PATTERNS:raise ValueError(Unknown input bayer pattern!)if not isinstance(raw, np.ndarray) or len(raw.shape) ! 2:raise ValueError(raw should be a 2-dimensional numpy.ndarray)if raw.shape[0] % 2 1 or raw.shape[1] % 2 1:raise ValueError(raw should have even number of height and width!)aug_pattern, target_pattern input_pattern, input_patternout rawif flip_h:out out[::-1, :]aug_pattern aug_pattern[2] aug_pattern[3] aug_pattern[0] aug_pattern[1]if flip_w:out out[:, ::-1]aug_pattern aug_pattern[1] aug_pattern[0] aug_pattern[3] aug_pattern[2]if transpose:out out.Taug_pattern aug_pattern[0] aug_pattern[2] aug_pattern[1] aug_pattern[3]out bayer_unify(out, aug_pattern, target_pattern, crop)return out4. Comments
初看就这用起来还挺香。没有很大的创新胜在工程价值较高。
Reference
[1] Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation [2] 官方代码
