Review of Inpainting Algorithms for Wireless Communication Application

Digital image inpainting is a technique of restoring large removed /damaged regions of an image with the data from the surrounding pixels of the removed region. The issue of image restoration with inpainting techniques occurs commonly in computer vision/image processing when unwanted objects have to be removed from images, for filling cracks in photographs, etc. Digital image inpainting approach is an active field of research in two significant applications of wireless communication: image compression and image recovery from a damaged image due to errors in a wireless channel. This work presents a brief survey of different image inpainting techniques and their contributions to different wireless communication applications. Keywords-inpainting; image compression; multipath fading; median filter; RS code


INTRODUCTION
Inpainting is a method of manually reconstituting paintings in a manner that the repaired region is imperceptible to the observer who is not acquainted with the image before damage. The term inpainting is lent from the arts, to describe a damaged picture recovery method. In this context, the inpainting paradigm may be considered from the following perspectives: • Inpainting for image processing solutions: The main goal is to restore removed or damaged regions propagating information from undamaged areas, e.g. restoring cracks in photographs, removing overlaid text on advertisements, occlusion recovery, etc.
• Inpainting based image coding: Signal processing solutions in which the main objective in exploiting digital inpainting techniques is to increase significantly the compression performance in comparison with standard coding solutions, e.g. JPEG and JPEG 2000 for images, and H.264/AVC (Advanced Video Coding) for video. Hybrid inpainting methods are those which include both inpainting and compression standards (e.g. JPEG).
• Restoration of an image received through fading channels: The intention of applying inpainting algorithms for image enhancement and restoration (which are important) is to interpret pictures received through fading channels more easily.
A literature study is presented in this paper, presenting inpainting algorithms for image restoration, image compression, and image recovery at the decoder side on images received through a multipath fading channel.

A. Partial Differential Equation (PDE) Image Inpainting
The first digital inpainting was based on diffusion inpainting technique, in which the missing information in an image region was completed by diffusing image data into the missing region after identifying the surrounding region. The aim was to design an automatic digital inpainting in order to reproduce the basic techniques employed by professional restorators. Basically, inpainting algorithms are based on the variational method and Partial Differential Equations (PDEs). Image inpainting algorithms built on PDEs use a mathematical model to treat image problems. By numerical representation, a PDE is iteratively solved to propagate the information smoothly from surrounding areas to the areas that are to be inpainted along isophotes. Isophotes are traces on which gray values are equal. The process of inpainting is terminated until the gray values in the region reach a steady state. Some early work related to inpainting, based on level lines, was carried out in [1]. The problem was considered as a disocclusion problem. T-junctions were defined in an edge map which they intersect the occluding objects. Digital image inpainting concept in completely digital sense was initiated in [2]. In this work, the interpolation method based on Laplacian was discussed. The propagated information along the isophotes' directions were estimated by the rotated image gradient by 90 degrees. Authors in [3] extended the work of [2] and proposed a variational method of inpainting based on PDEs. The PDEs were devised and the solutions obtained were based on level sets of the intensity function. A method based on the outlines of the Navier-Stokes was introduced in [4]. The major drawback of [2][3][4] was that their inability to reconstruct regions containing textures. Inpainting methods like those require complex energy functionals and solutions were found from the assumptions of Bounded Variation (BV) model. For non-texture images, theoretical image models popularly known as the TV (Total Variation) model were adopted in [5], in which the TV algorithm was extended to the Curvature-Driven Diffusion (CDD) model. Every PDE-based inpainting model is more

B. Texture Synthesis and Exemplar based Image Inpainting
Anisotropic diffusion is related with the inpainting field [6] and keeps the boundaries intact across the inpainted region to avoid the edges getting blurred. When focusing on small regions, simpler models can be considered instead of specific mathematical geometrical theories. One such method [7] performs convolution using a simple 3×3 coefficient filter on the missing region to diffuse valid/known information to the missing domain pixels. To solve this problem, a diffusion barrier must be considered to stop the algorithm, where the changes in contrast are more significant. To remove film scratches in a video, the method based on spacio-temporal inpainting and proposed in [8] restores the image pixels in scratch areas by considering image pixels of a undamaged region in the same image frame and also considering the frames which are adjacent to it. Texture synthesis is a technique related to inpainting. Basically it deals with selecting and synthesizing the texture into the domain of inpainting. The texture's most inherent property is its regularness. A regular (deterministic, periodic, structured) texture is characterized by a basic element (texel or texeton) which is regularly placed on lattice or a grid. Examples of regular textures are floor tiles, brick walls, and sand while smoke is non-regular.
Every texture contains some deterministic and stochastic properties. Texture synthesis algorithms test a variety of textures from periodic to most random types. In 1999, authors in [9] found a different approach to handle texture synthesis. They adopted a nonparametric way of developing texture from an initial seed. This was the first kind of image-based texture synthesis. They copied pixels of the image itself in the synthesis process. In [10], a specific algorithm was proposed to synthesize natural textures. In raster manner, every pixel has four previously synthesized neighbor pixels. The algorithm reduces the search space of every pixel to only four neighbors. If the left neighbor pixel is synthesized from (x; y) on the sample, its candidate pixel is (x+1; y). These methods favored copying neighbor pixels to preserve larger texture structures. It was considered as an intermediate step just before the development of patch based synthesis. They gave better and faster results for structure related problems. The first patchbased texture synthesis algorithm developed was image quilting [11], in which the sample of the image is treated as grid with a block size of 32×32, with 4 to 8 pixels overlapping each other by adjacent blocks. The blocks are picked up from the image sample texture to fit in a raster scan. In the search process, regions that are overlapping on left and top of a new block are matched. Authors in [12] sought for better quality in reducing errors by finding boundaries between patches. Graphcut synthesis procedure is not fast, but it provides very good results. Variational inpainting and texture synthesis algorithms are separate fields of research, though their intention is to solve related problems. Inpainting aims at filling an inpainting region by preserving image smoothness, texture synthesis generates new texture from a sample.
Authors in [13] carried out image completion by applying structure inpainting and texture synthesis simultaneously to fill image holes. In their work, image decomposition divides the image into structured regions and texture parts. Inpainting is performed on the structure image and texture synthesis on the texture image. The two images are recombined to get the final result. The methods in [2,9] are appropriately applied for damaged regions, and the final result is obtained by adding both. The algorithm is a bit slow, gives blurry results due to diffusion, and is limited to small gaps. However, it is the first pioneered work that combines the strengths of both structural inpainting and texture synthesis. Similar efforts were made in [14], based on adaptive ND tensor voting. It provides good results, and large damaged areas can be repaired, but the technique requires the implementation of image segmentation. The most exquisite contribution in this field is a work popularly known as exemplar-based image inpainting [15] in which an sampling patch-based inpainting method for region filling and object removal method which is simpler and faster was proposed. Inspired by [9], it selects locations at boundary gaps, then image blocks are matched with the copies searched from known regions to complete the gap. The method performs the synthesis task through a best-first filling strategy that depends entirely on the priority values assigned to each patch. After finding the patch with the maximum priority, the most similar patch is chosen from the intact region to replace it, and then the priority values are updated to continue the above steps repeatedly.
Except from exemplar-based methods, the following methodologies were applied to image inpainting and satisfactory results were obtained. Authors in [16] applied discrete global optimization technique which was timeconsuming. An image inpainting based on context-aware patch-based was presented in [17]. The image was divided into patches with variable sizes. This encoding method maintained the consistency of neighboring patches. To complete the large gaps surrounded by varieties of structure and texture, authors in [18] achieved both significant speed and performance. To limit patch selection, a method on patch subspace learning which was sensitive to the initial values was proposed in [19]. To enhance the similarity between matching patches, runtime is reduced by the coherent direction-aware patch alignment scheme [20]. A patch-based image completion [21] with patch transformation guides the filling process. From the works available on inpainting, PDE based interpolation inpainting technique provides a simple and appropriate solution for filling the missing region's information while texture information can be used to grow the lost texture data. A state-of-the-art method which uses dense multi-scale fusion network with self-guided regression loss and geometrical alignment constraint for image fine-grained inpainting which highly improves the quality of produced images is proposed in [22].

III. INPAINTING ALGORITHMS FOR IMAGE COMPRESSION
Inpainting techniques exploit the redundancy which is inherent in images and achieve lossy compression. They aim at finding very compact representations in such a way that the results approximate the original image. Some prominent characteristics extracted from the images are skipped at the transmitter side. At the receiving end, they are recovered after using assisted inpainting techniques. The scheme in [23] is based on the framework of image patches for compression. From the compression point of view, a huge set of image patches can be represented with different shapes, sizes, and orientations. However, the restricted patches must be regular ones, e.g. squares with a fixed size. This idea stems from the texture synthesis technique, in which the texture is synthesized by sampling the best match patch from the known region. The reconstruction is only required for the removed regions by virtue of the preserved ones, which is virtually the inpainting problem carried out at a patch level to complete the image.
Several PDE-based and texture synthesis image inpainting techniques in achieving image compression are discussed in [24][25][26][27][28]. PDE-based methods are alternatives to transformbased methods such as the JPEG standard [29] which employs Discrete Cosine Transform (DCT), or its successor JPEG 2000 [30] that uses wavelet transform. In [31], implementing inpainting for compression is explained. In this approach, image regions dropped at the encoder intentionally describe the concise image. It also elaborates the transmission of image over multipath fading channels and reconstruction details about filling of the damaged or missing blocks in the propagation of compressed images (JPEG). Since it is difficult for inpainting methods to recover large regions that were dropped, edgebased inpainting techniques were developed for image compression [32,33]. Edge information is extracted from dropped regions and later is converted into a compressed form which assists the inpainting process at the decoder. By recovering more edge regions, the edge-based inpainting achieves higher compression ratio. Authors in [34] address an adaptive patch-based inpainting algorithm for recovering image blocks in block-based image coding. In order to achieve compression, a generic Parameter Assistant Inpainting (PAI) method exploits different distributions of an image region in [35]. A sparse representation of these parameters assists reliable inpainting at the decoder side to recover large dropped regions. The inpainting techniques proposed in [36,37] have the ability to reproduce the lost information faithfully. This enables the user to intentionally or voluntarily remove some regions in an image, i.e. by providing the user the ability to select the regions that he wishes to remove. Large dropped regions are restored utilizing the correlations between image blocks. To continue to achieve high CR and the required level of visual quality, a hybrid image compression method which embeds inpainting schemes into current transform based compression standards is considered to get the benefits of both algorithms. In [37], large dropped areas are recovered by filling various incomplete image regions, which consists of completely different descriptions of structures and surrounding backgrounds. This method achieves compression ratio up to 50%.
A recent work on image compression with image inpainting based on PDE approach concentrated on spatial and tonal data optimization has been published in [38]. This paper also includes a comprehensive survey on PDE-based image compression. Although PDE-based methods are an alternative solution to transform-based methods such as the JPEG standard, from the earlier research publications, the conclusion can be drawn that combination of PDE based inpainting and JPEG provides better solutions and wider range of compression ratios. The PDE approach concentrated on spatial and transform-based compressions provides better results in terms of compression ratio without any need of assistant parameters. Multi-scale convolutions that sample content at multiple scales to assist with inpainting are introduced in [39]. The proposed inpainting and compression models are jointly trained to show that inpainting reduces the amount of information.

IV. IMAGE TRANSMISSION OVER MULTIPATH FADING CHANNELS AND RECONSTRUCTION
In digital wireless communication systems, the multipath channel reflects signal at multiple places. Consequently, the signal traverses to reach the receiver along several paths in addition to the direct signal. The channel impairments influence the transmission phenomenon in such a manner that errors occur in bursts and the damaged pixels become inconsistent with their adjacent pixels which are received correctly. From the reports in literature, the average loss rate of packets in propagation environment is 3.6% and they occur in a bursty manner. Critically, we might lose an entire line of an image. When image or video data are transmitted over noisy channels, some data are either missing or incorrect due to channel transmission errors or channel impairments. Images are prone to impulse (stuck-pixel) noise and other interference noises. Based on the impulse noise pixel's value, it can be either categorized into Fixed Valued Impulsive Noise (Salt and Pepper Noise -SPN) or Random Valued Impulsive Noise (RVIN). In suppressing impulse noise, filters designed with median, minimum, and maximum are special types of orderstatistics which are non-stationary (shift-variant) and nonlinear. The contribution of inpainting algorithms based on variants of median filter deals with the removal of two types of impulse noise. These filter structures (Adaptive Median Filter and Directional Weighted Median) are capable of restoring images that are affected by both RVIN and SPN under extreme conditions and provide adequate results in the regions containing both slow variations in gray and high contrast edges. Many algorithms have been proposed in the past to remove impulse noise. Authors in [40] deal with the basics of median filter, the development of a weighted median filter and the theory of optimal weighted median filter for speech processing, weighted medians as robust predictors in DEP coding and Quincunx coding for image sequence restoration. The technique reported in [41] efficiently restores images which are corrupted due to mixed impulse and Gaussian noise. In [42], the median based filters are widely used because of their effective noise suppression capability and high computational efficiency. The technique uniformly replaces the gray-level value of every pixel by the median of its neighbors in the sliding window. Consequently, some desirable details are also removed, in particular, when the window size is large. For the removal of random-valued impulse noise, the Directional Weighted Median (DWM) filter [43] and the Multi-stage Directional Median (MSDWM) filter [44] provide better solutions. The filter presented in [45] attempts to eliminate SPN effectively. The filter in [46] uses a new impulse detector, which is based on the difference between the current pixel and its neighbors aligned with four main directions. The impulse detector addresses the challenge of detecting noisy samples and the output of noise removal technique is applied only to the affected image pixels. The algorithm is fast and provides good results in both edges and smooth regions. It is also understood from [47] that different kinds of errors such as SPN and RVIN can be removed from the variants of median filters such as AMF and DWMF by employing the surrounding information of the lost data. By diffusing the median information of pixels from the outside to the inside of the inpainted region, the important properties of edges are preserved. Bilateral filtering, produces unsatisfactory results. To process edge pixels and image pixels separately, Rank Ordered Absolute Differences (ROAD) technique is incorporated by adding a third component to the weighting function and the resulting filter is a trilateral filter. The filter in [48] preserves the edge details.
An inpainting algorithm based on Directional Weighted Median (DWM) filter/ Adaptive Median Filter [49] de-noises the affected pixels (RVIN or SPN) when applied on images received through multipath fading channels (Rayleigh and Rician). The algorithm diffuses the median value of pixels from the exterior area into the inner area to be inpainted and thus preserves edges and fine details. A unique noise removal algorithm is proposed in [50] which detects up to 75% noise density using region based detection and filters. Progressive image transmission is difficult in the presence of a noisy channel, mainly due to the propagation of errors during the decoding of a progressive bit stream. Excellent results for this problem can be obtained through combined source channel coding techniques [51][52][53]. In [54,55], the authors discuss the image transmission approach and demonstrate the performance of the scheme when implemented over mobile fading channels. A two-stage strategy is described. The first stage comprises of Error Control Coding (ECC) and the second of an inpainting tool for image reconstruction. Visually plausible images are obtained from the combination of these two stages over a range of Doppler frequencies. In [56], an inpainting algorithm applied to recover the errors from compressed data transmission through Wireless Sensor Networks (WSNs) has been proposed. The algorithm's efficacy to restore the missing information of lossy received images has been experimentally proven. Authors in [57] propose a method for completing lost blocks in H.264 compressed image transmission. The losses can be recovered from inpainting and the synthesis of textures combination provided that the features are not completely lost in the received image. In [58], Copy-and-Paste Network, based on Deep Neural Network framework, video inpainting is performed by utilizing the additional information from other video frames. The network is trained in such a way that it copies the contents in reference frames and pastes them to fill the gaps/holes in the target frame. The material available in [59] is helpful for channel implementation and in creating various multipath fading channel models. To compare ECC, details related to Forward Error Correcting code such as Reed Solomon code are referred from [60].

V. MODELLING MULTIPATH FADING WIRELESS CHANNELS
Two kinds of mobile fading effects describe the channel characteristics: large and small scale fading [60,62]. Rayleigh or Rician fading are examples of small-scale fading. When multiple reflective paths are encountered, the received signal envelope is expressed through a Probability Density Function (PDF) related to a Rayleigh or a Rician distribution [62]. Author in [63] summarized eight descriptions regarding Rayleigh fading channel simulations consisting of one deterministic model and seven statistical models. To model the Rayleigh fading channel, the models in [64,65] have been widely used. The software simulations to model multiple path fading channels are important components of wireless systems simulations. Authors in [66] discuss various ways of modeling fading channels (flat) in MATLAB. The details are helpful in modeling many of the fading channel effects with different test conditions. VI. CONCLUSION In this paper, the contributions of pure inpainting are reviewed. A detailed review of PDE-based image compression methods which constitute a challenging application of inpainting algorithms is conducted. Several variants of filtering techniques are studied for impulsive noise removal. Error control coding technique was selected for comparison. The review work embodied in this paper addressed two problems associated with wireless communication and highlighted restoration algorithms to find the solutions. The two problems are: image compression to achieve reduced bit rate (increase in compression ratio) using hybrid compression method and image propagation over multipath fading channels followed by recovery of pixels damaged due to channel effects.