review article literature survey on stereo vision

Review ArticleLiterature Survey on Stereo Vision Disparity Map Algorithms

Rostam Affendi Hamzah12 and Haidi Ibrahim1

1School of Electrical amp Electronic Engineering Universiti SainsMalaysia Engineering Campus 14300 Nibong Tebal Penang Malaysia2Fakulti Teknologi Kejuruteraan (FTK) Universiti Teknikal Malaysia Melaka Kampus Teknologi Hang Tuah Jaya76100 Durian Tunggal Melaka Malaysia

Correspondence should be addressed to Haidi Ibrahim haidi ibrahimieeeorg

Received 21 August 2015 Accepted 11 November 2015

Academic Editor Eduard Llobet

Copyright copy 2016 R A Hamzah and H Ibrahim This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited

This paper presents a literature survey on existing disparitymap algorithms It focuses on fourmain stages of processing as proposedby Scharstein and Szeliski in a taxonomy and evaluation of dense two-frame stereo correspondence algorithms performed in 2002To assist future researchers in developing their own stereomatching algorithms a summary of the existing algorithms developed forevery stage of processing is also providedThe survey also notes the implementation of previous software-based and hardware-basedalgorithms Generally the main processing module for a software-based implementation uses only a central processing unit Bycontrast a hardware-based implementation requires one or more additional processors for its processingmodule such as graphicalprocessing unit or a field programmable gate array This literature survey also presents a method of qualitative measurement thatis widely used by researchers in the area of stereo vision disparity mappings

1 Introduction

Computer vision is currently an important field of researchIt includes methods such as image acquisition processinganalysis and understanding [1] Computer vision techniquesattempt tomodel a complex visual environment using variousmathematical methods One of the purposes of computervision is to define the world that we see based on one ormore images and to restructure its properties such as itsillumination shape and color distributions Stereo vision isan area within the field of computer vision that addresses animportant research problem which is the reconstruction ofthe three-dimensional coordinates of points for depth esti-mation A system of stereo vision system consists of a stereocamera namely two cameras placed horizontally (ie one onthe left and the other on the right) The two images capturedsimultaneously by these cameras are then processed for therecovery of visual depth information [2] The challenge is todetermine the best method of approximating the differencesbetween the views shown in the two images to map (ieplot) the correspondence (ie disparity) of the environment

Intuitively a disparity map represents corresponding pixelsthat are horizontally shifted between the left image andright image New methods and techniques for solving thisproblem are developed every year and exhibit a trend towardimprovement in accuracy and time consumption

Another device that is used to acquire depth informationis a time-of-flight (ToF) or structured light sensor Such adevice is a type of active sensor unlike a classic stereo visioncamera Devices of this type such as the Microsoft Kinectare cheap and have led to increased interest in computervision applications However these active sensors suffer fromcertain characteristic problems [3] First they are subject tosystematic errors such as noise and ambiguity which arerelated to the particular sensor that is used Second theyare subject to nonsystematic errors such as scattering andmotion blur According to the comparative analyses per-formed by Foix et al [4] Kim et al [5] and Zhang et al[6] ToF devices perform satisfactorily only up to amaximumdistance of approximately 5ndash7 meters and are too sensitive tobe used in outdoor environments especially in very brightareas Because of these limitations of ToF sensors stereo

Hindawi Publishing CorporationJournal of SensorsVolume 2016 Article ID 8742920 23 pageshttpdxdoiorg10115520168742920

2 Journal of Sensors

Quantity of original papers from ScienceDirect

22 23 39

32

33

43

50

58

41

47

YearTrendline

0

10

20

30

40

50

60

70

Qua

ntity

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

Journalbook

(a)

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

YearTrendline

Quantity of original papers from IEEE Xplore

92

144

144

165

198

207

186

224

168

171

0

50

100

150

200

250

Qua

ntity

Conferencejournal

(b)

Figure 1 The yearly distributions of the number of papers published from 2005 to 2014 in the area of stereo vision from the database of (a)ScienceDirect and (b) IEEE Xplore

vision sensors (ie passive sensors) are more reliable androbust they are able to produce high-resolution disparitymaps and are suitable for both indoor and outdoor environ-ments [7]

In stereo vision disparity map processing the num-ber of calculations required increases with an increasingnumber of pixels per image This phenomenon causes thematching problem to be computationally complex [8] Theimprovements to and reduction in computational complexitythat have been achieved with recent advances in hardwaretechnology have been beneficial for the advancement ofresearch in the stereo vision field Thus the main motivationfor hardware-based implementation is to achieve real timeprocessing [9] In real time stereo vision applications suchas autonomous driving 3D gaming and autonomous roboticnavigation fast but accurate depth estimations are required[10] Additional processing hardware is therefore necessaryto improve the processing speed

An updated survey on stereo vision disparity map algo-rithms would be valuable to those who are interested in thisresearch area Figures 1(a) and 1(b) illustrate the quantityof original contributions published in this area over thepast ten years (ie 2005ndash2014) from the databases of Sci-enceDirect and IEEE Xplore The keywords used were stereovisionstereo vision algorithm and the components that weresearchedwere the title abstract and keywordsindex terms ofthe papers in the databases All of these papers may representcontributions to fundamental algorithm development analy-sis or application of stereo vision algorithms In both figuresthe trendlines are increasing indicating that the field ofstereo vision remains active in research and development andhas become an interesting and challenging area of researchThis paper provides a brief introduction to the state-of-the-art developments accomplished in the context of suchalgorithms This work reviews the latest published stereovision algorithms and categorizes them into different stagesof processing which are based on the taxonomy proposedby Scharstein and Szeliski [11] This paper also discusses

two types of implementation platforms for these algorithms(ie software-based and hardware-based) In software-basedplatforms the techniques are implemented only on a standardCPU without any other additional processing hardware Incontrast to hardware-based platforms the algorithms areexecuted on a CPU with a GPU or FPGA as a standalonesystem

The remainder of the paper is organized as followsPrevious review papers related to stereo vision disparity mapalgorithms are discussed in Section 2Then the taxonomy forthe stages of processing performed in stereo vision disparitymap algorithms is presented in Section 3 It consists of foursubsections (ie matching cost computation cost aggre-gation disparity selection and optimization and disparityrefinement) Section 4 presents a review of algorithms imple-mented through software-based platforms and Section 5discusses real time stereo vision disparity map algorithmsbased on additional hardware (ie FPGAs and GPUs) Amethod ofmeasuring the accuracy of stereo vision algorithmsis explained in Section 6 and the conclusion is presented inSection 7

2 Previous Reviews of Stereo VisionDisparity Map Algorithms

Numerous methods of implementation for stereo visiondisparity mapping have been established in the past fewyears This can be observed from the review papers listedin Table 1 The contents of these review papers are alsosummarized in this table Among these review papers themain focus was to summarize and compare the accuracylevel and execution time of each cited algorithm Howevernone of these reviews provided a detailed discussion of thestages of implementation based on the taxonomy proposedby Scharstein and Szeliski [11] as does the survey presentedin this paper Furthermore this paper also reviews thelatest algorithms implemented using two different types ofplatforms (ie software-based and hardware-based)

Journal of Sensors 3

Table 1 Previous review papers on stereo vision disparity map algorithms

Year Author Focus

2002 Scharstein and Szeliski [11]

Proposed a taxonomy for vision algorithms and provided a quality metric to compare andevaluate multiple blocks of algorithms as shown in Figure 1 They have also provided a test bed formeasurable evaluation of stereo depth map algorithms The test bed or benchmarking datasetconsists of four images (Tsukuba Venus Teddy and Cones) which are available athttpwwwmiddleburyedustereo

2003 Brown et al [19] Reviewed advances in stereo vision disparity map algorithms regarding correspondence methodsand occlusion handling methods for real time implementations

2008 Tombari et al [77] Presented a survey and compared the different methods of cost aggregation for stereocorrespondence through accuracy and computational requirements

2008 Lazaros et al [12]Reviewed developments in stereo vision algorithms implemented via software and hardwarecategorized in terms of their major attributes The comparison of local and global methodsprovided by previously developed algorithms implemented on software and hardware basedplatforms was presented in this work

2011 Tombari et al [101] Contributed an evaluation of stereo vision depth map algorithms in terms of their 3D objectrecognition ability

2013 Tippetts et al [8]Reviewed stereo vision algorithms and their suitability for resource-limited systems They havecompiled and presented an accuracy and runtime performance data for all stereo vision disparitymap algorithms in the past decade with an emphasis on real time performance

Inputimages

Step 1

Matchingcost

computation

Step 2

Costaggregation

Step 3

Disparityselection

Step 4

Disparityrefinement

Disparitymap

Figure 2 A framework for the development of a stereo vision algorithm

3 A Taxonomy for the Processing Stages ofStereo Vision Disparity Map Algorithms

Most stereo vision disparitymap algorithms have been imple-mented using multistage techniques These techniques ascodified by Scharstein and Szeliski consist of four main stepsas shown in Figure 2 [11] In this figure the input images areobtained from stereo vision sensors (ie from at least twocameras) Commonly these cameras are arranged horizon-tally and set up which produce two or more correspondingimages For the explanation or the process as described byadopted taxonomy these input images are assumed to berectified images Next the image pair to be analysed will passthrough all of the blocks in sequence beginning with Step1 and ending with Step 4 The output of this process shouldbe a smooth disparity map In essence each block representsone ormore algorithms whose performance can bemeasuredbased on the expected outputThis taxonomyhas beenwidelyused by many current developers of stereo vision disparitymap algorithms [8 12]

In general stereo vision disparity map algorithms can beclassified into local or global approaches A local approachis also known as area based or window based approachThis is because the disparity computation at a given point(or pixel) depends only on the intensity values within apredefined support window Thus such method considersonly local information and therefore has a low computationalcomplexity and a short run time Local methods include all

four steps of the taxonomy Examples of implementation ofsuch methods are provided by the work of Mattoccia et al[13] Arranz et al [14] and Xu et al [15] The disparity mapvalue assignment is achieved through winner take all (WTA)optimization For each pixel the corresponding disparityvalue with the minimum cost is assigned to that pixel Thematching cost is aggregated via a sum or an average over thesupport window

By contrast a global method treats disparity assignmentas a problem of minimizing a global energy function forall disparity values Such a method is formulated as anenergy minimization process with two terms in the objectivefunction (ie a data term which penalizes solutions thatare inconsistent with the target data and a smoothness termwhich enforces the piecewise smoothing assumption withneighboring pixels) The smoothness term is designed toretain smoothness in disparity among pixels in the sameregion The disparity map is produced by assigning similardepth values to neighboring pixels Global methods producegood results but are computationally expensive Thereforethey are impractical for use in real time systems Globalmethods typically skip Step 2 of the taxonomy depicted inFigure 2 (ie they do not perform cost aggregation andtherefore contain only three steps) [16ndash18] Markov randomfield (MRF) modelling is the approach that is most commonapproach used in globalmethodsThis type ofmodelling usesan iterative framework to ensure smooth disparity maps andhigh similarity between matching pixels


P

Il Ir

pl pre e

OlOr

Figure 3 Epipolar geometry The 3D image of target scene at point119875

31 Matching Cost Computation All stereo matching algo-rithms require a cost criterion tomeasure the extent ofmatch-ing between two pixelsThematching cost computation is thestage in which whether the values of two pixels correspond tothe same point in a scene is determinedTherefore the stereomatching cost computation can be defined as a method ofdetermining the parallax values of each point between the leftand right images [19] The matching cost is computed at eachpixel for all pixels under considerationThe difference in pixelintensity between a pair of the matching pixels in two imagesis called the disparity and can be associated with depth valuesthrough three-dimensional (3D) projection

The matching points must lie on epipolar lines 119890 asshown in Figure 3 This matching can be performed via aone-dimensional horizontal search if the stereo pairs areaccurately calibrated [20] A target point 119875 is viewed fromthe optical centers 119874119897 and 119874119903 of the two cameras It producesone left image plane 119868119897 and one corresponding right imageplane 119868119903 one from each of the two cameras The points 119901119897and 119901119903 represent the matching pixel intensities in the left andright image planes respectively for the same scene at point119875Therefore the targeted left and right matching points mustbe located at the same coordinates on the horizontal line 119890In a stereo vision system this requirement is imposed in theform of an epipolar constraint [21] This epipolar constraintplays a significant role in stereo matchingThis is because thesearch for correspondences can be limited to a line insteadof the entire image space thereby reducing the required timeand search range Early stereo vision disparity map algorithmfor the matching cost computation task uses pixel basedtechnique [12] These algorithms are the methods of absolutedifferences (AD) squared differences (SD) adaptations ofthe former include the methods of sampling-insensitiveabsolute differences and truncated absolute differencesThesealgorithms can be applied to grayscale or color images

Area based or window based techniques are capableof offering richer data than matching techniques basedon individual pixels or features Such techniques can be

more accurate because the matching process considers theentire set of pixels associated with image regions Commonalgorithms for window based techniques include the sum ofabsolute differences (SAD) the sum of squared differences(SSD) normalized cross correlation (NCC) rank transforms(RT) and census transforms (CT) [12] The matching cost iscalculated over a support region This support region whichis commonly referred to as support or aggregating windowmay be square or rectangular and may be fixed or adaptivein sizeThemajor shortcoming of window based technique isthat these approaches commonly assume that all pixels withina support window have similar disparity values This is notnecessarily true for pixels near depth discontinuities or edgesHence an improper selection of the size and shape of thematching window can lead to poor depth estimations

311 Absolute Differences (AD) The AD algorithm aggre-gates the differences in luminance (or intensity values)between the pixel in the left image 119868119897 and the correspondingpixels in the right image 119868119903 as given by

AD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (1)

In this equation (119909 119910 119889) represents the disparitymap coordi-nates where (119909 119910) are the coordinates of the pixel of interestand 119889 is the disparity (or depth) value Typically in thematching process 119868119897 is used as the reference image and theright image 119868119903 represents the target (or candidate) imageThe AD algorithm is the simplest among matching cost algo-rithms Because of its low complexity Wang et al [22] usedthis algorithm for real time stereo matching using graphicshardware (GPU) The AD algorithm functions satisfactorilyin regions with little texture but for highly textured imagesthis algorithm is not capable of producing a smooth disparitymap To overcome this difficulty the truncated version ofthe AD algorithm was developed The truncated absolutedifference (TAD) algorithm as implemented by Min et al[23] and Pham and Jeon [24] is able to minimize the errorsin disparity maps Furthermore the TAD algorithm usesthe colors and gradients at matching pixels to improve itsrobustness against variations in illumination

312 Squared Differences (SD) The SD algorithm aggregatesthe squared differences between the reference pixels in 119868119897 andthe candidate pixels in 119868119903 as described in

SD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (2)

Yang et al [25] implemented the SD algorithm for theirmatching cost computation in a subpixel estimation methodfor disparity mapping Considerable noise was generated atthe boundaries in their initial disparity maps However Yanget al applied a bilateral filter (ie a type of edge preservingfilter) to improve the flattening of edges and to smooth areasnear depth discontinuities Recently Miron et al [26] testedvarious matching cost functions in their stereo disparitymap algorithms for intelligent vehicle applications Theyconcluded that the SD algorithm produced the largest errorThe errors that occur due to the SD algorithm are highlysensitive to brightness and noise especially in real timeenvironment


313 Feature Based Techniques The matching cost func-tion can also be constructed using feature based tech-niques This approach attempts to establish correspondencesonly for similar feature points that can be unambiguouslymatched Common methods of feature extraction includethose based on visual features (eg edges shapes texturessegmentation and gradient peaks) statistical characteristics(eg minima medians and histograms) and transformationfeatures (eg Hough transforms wavelet transforms andGabor transforms) [27] As an example Sharma et al [28]developed a new disparity map algorithm using featuresderived from the scale invariant feature transform (SIFT)for autonomous vehicle navigation In their implementationthey modified the SIFT algorithm to use self-organizing mapto achieve more efficient performance in feature matchingprocess Their results indicate that the computation time oftheir method is reduced compared with the conventionalSIFT algorithm Because only feature points are correlatedalthough the computational cost is significantly reducedthe complete disparity map cannot be obtained The featurematching accuracy is still low Sparse disparity maps areproduced because only matching points derived from thetargeted object features are used [29] Furthermore featurebased matching techniques exhibit low accuracy and arerather insensitive to occlusion and textureless areas Liuet al [30] used a combination of image segmentation andedge detection for the matching cost function The timeimplementation was fast but the accuracy level also remainedlow in regions of discontinuity Ekstrand et al [31] used asegmentation process that created one-dimensional segmentscontaining information on color and edge coordinates Asegment correspondence estimation was performed for everyrow of the image to reduce the possibility of mismatchedsegments However errors still occurred as a result of usinglimited vertical support segments during the matching pro-cess Thus feature based techniques are not preferred andtheir usage by researchers for the development of disparitymap algorithms remains low

314 Sum of Absolute Differences (SAD) TheSADalgorithmdefined in (3) considers the absolute difference between theintensity of each pixel in the reference block and that of thecorresponding pixel in the target block

SAD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (3)

Thedifferences are summed over the aggregated support win-dow 119908 to generate a simple metric of block similarity knownas a disparity map The SAD algorithm is a well-known algo-rithm for matching cost computation The SAD algorithm isable to function in real time implementation because of itslow computational complexity This was proven by Tippettset al [32] who calculated and evaluated SAD performancesfor real time human pose images in a resource limited systemLee and Sharma [33] implemented real time disparity mapalgorithm estimations using the sliding window techniqueto calculate matching costs using the SAD algorithm Theiralgorithm uses parallel processing via a graphical processing

unit (GPU) By virtue of applying this new technique at thematching cost stage the accuracy of stereo vision process-ing can be increased while simultaneously improving thespeed

Gupta and Cho [34] implemented a new techniqueusing two different sizes of correlation windows in the SADalgorithm At the first level the initial cost aggregation isdetermined and at the second level the object boundariesare improved using a smaller window size compared withthe first-level implementationThe results produced aremoreaccurate in terms of pixel matching but still require theimplementation of multiple loops at different window sizesThe SAD algorithm is fast but the quality of the initialdisparity map that is produced is low because of the noise atobject boundaries and in textureless regions

315 Sum of Squared Differences (SSD) Equation (4) pre-sents the SSD algorithm in which the summation is per-formed over the squared differences in pixel intensity valuesbetween two corresponding pixels in the aggregated supportwindow 119908

SSD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (4)

An early implementation of the SSD algorithm for thematching cost calculation stage was achieved by Fusielloet al [36] They tested the SSD algorithm on multiple fixedwindow blocks to reduce the incidence of occlusion errorsThe purpose of using multiple blocks of windows is tosearch for the smallest error to select an appropriate pixelof interest in the disparity map Yang and Pollefeys [37]used technique similar to that presented in [36] but theiralgorithm was implemented on platform with a GPU Theyachieved good results in terms of speed compared withFusiellorsquos work Currently there is still relatively little researchon the use of the SSD algorithm in for stereo vision disparitymap algorithms compared with that on other matching costalgorithms This is evident from the previous review papers[8 12] on stereo vision disparity map algorithms and is alsoshown in Table 2

316 Normalized Cross Correlation (NCC) The NCC algo-rithm is another method of determining the correspondencebetween two windows around a pixel of interest The nor-malization within the window compensates for differences ingain and bias [38] Equation (5) specifies the formula for theNCC technique

NCC (119909 119910 119889)

=

sum(119909119910)isin119908119868119897 (119909 119910) sdot 119868119903 (119909 minus 119889 119910)

radicsum(119909119910)isin119908 1198682

119897(119909 119910) sdot sum

(119909119910)isin1199081198682119903(119909 minus 119889 119910)

(5)

However the NCC algorithm tends to blur regions of dis-continuity more than other matching cost algorithms [38]This is because any outliers lead to large errors in the NCCcalculations A new method for low-dimensional image fea-tures matching using NCC has been proposed by Satoh [39]


Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash

Journal of Sensors 7Ta

ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking

Multiresolutionadaptiv

ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization

mdashMultiresolutionenergy

minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC

mdashHierarchicalbilateral

disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion

Multiresolutionaniso

tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM


The NCC algorithm was chosen because of its robustness tointensity offsets and changes in contrastThe results achievedin Satohrsquos work exhibit high accuracy but considerablecomputational resources are required Additionally Chenget al [40] implemented their matching cost calculation usinga zero mean normalized cross correlation (ZNCC) in whichthe pixels at which edges are located are manipulated via amultiple-window strategy This method relies on a neutralnetwork model Furthermore the least-mean-square deltarule is used for training and for the determination of theproper window shape and size for each support regionThesetechniques offer improved accuracy but their computationalrequirements are still high

317 Rank Transform (RT) Thematching cost for RT is givenby (6) and is calculated based on the absolute differencebetween two ranks (ie Rankref from the reference image andRanktar from the target image)

RT (119909 119910 119889)

= sum

(119909119910)isin119908

1003816100381610038161003816Rankref (119909 119910) minus Ranktar (119909 minus 119889 119910)1003816100381610038161003816

(6)

In this equation Rankref and Ranktar are calculated as shownin

Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)

with 119871(119894 119895) as defined in

119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)

where (119894 119895) are the coordinates of a neighboring pixel and(119909 119910) are the coordinates of the pixel of interest Equation(8) computes the number of neighboring pixels 119868(119894 119895) thathave values larger than that of the central pixel 119868(119909 119910) Anew model of a disparity map algorithm that uses the RTalgorithm to achieve improved accuracy has been proposedby Gac et al [41] Through careful selection of the windowsizes a reliable initial disparity map is efficiently obtainedAt the time of its publication this method demonstrated thehighest correct initial matching rate of any matching costalgorithm in date The RT algorithm is typically effectivefor coping with brightness differences and image distortionsSometimes when the RT algorithm is used a matching pixelmay look extremely similar to a neighboring pixel leadingto matching ambiguity In [42] a new extension of the RTapproach was developed to reduce this matching ambiguityusing a Bayesian model This model considers not only thesimilarities between the left and right image pixels similaritiesbut also the level of ambiguity within each image inde-pendently The results of experiments on images exhibitingvariations in intensity and brightness differences as reportedby authors of that study indicate a reduction in matchingambiguities

318 Census Transform (CT) The CT algorithm translatesthe results of comparisons between a center pixel and itsneighboring pixels within a window into a bit string as shown

Census (119909 119910) = Bitstring(119894119895)isin119908(119868 (119894 119895) ge 119868 (119909 119910)) (9)

This algorithm is calculated using the Hamming distancesbetween the census bit strings of on the correspondingmatchcandidates as given by

CT (119909 119910 119889) = sum(119909119910)isin119908

Hamming (Censusref (119909 119910)

minus Censustar (119909 minus 119889 119910)) (10)

where Censusref represents the census bit string from thereference image andCensustar represents the census bit stringfrom the target image The CT algorithm is rather robustto the disparity discontinuities because of its good outliertolerance as described byHumenberger et al [43]This claimwas proven by performance comparisons between the CTalgorithm and the SAD algorithm The disparity maps pro-duced by the CT algorithm exhibited highermatching qualityat object borders than those produced by the SAD algorithmThe disadvantage of the CT algorithm is its tendency to pro-duce incorrect matches in regions with repetitive structuresThis shortcoming was mitigated by Ma et al [44] throughtheir modifications to the CT algorithm They implementedadditional bits to represent the differences between the pixelof interest and the neighborhood pixels According to theirresults the accuracy of the disparity map was improved andthe incorrect matching problem was alleviated by this mod-ification In addition the proposed algorithm demonstratedgreater robustness when applied to a noisy image comparedwith the conventional CT algorithm

Several researchers have also developed matching costmethods based on a combination of two algorithms A com-bination of the AD and CT algorithms as shown by Mei et al[45] successfully reduces the occurrence of errorsThe reasonfor combining these two methods is to compensate for theirrespective limitations The CT algorithm tends to produceincorrect matches in regions with repetitive local structureswhereas the AD algorithm does not performance well onlarge textureless regions Similarly a combination of the SADand CT algorithms will also lead to higher performance butwill incur an increase in computational complexity [46] TheSAD and CT cost measures are obtained individually and thefinal cost function is constructed as a linear combination ofboth cost measures based on a weighting factorThe accuracyimprovement achieved by Zhang et al [47] was accomplishedbymeans of a costmeasure combining the SAD approach andarm length differences (ALD) The use of ALD was inspiredby the similarity of the matching pixels support regions inthe vertical direction as a result of the pixels being locatedon the same horizontal line This combination is able toreduce errors in most regions especially those containingrepeated color and shapes Lee et al [48] combined the CTand gradient difference approaches to achieve a highermatching cost quality However according to themmatchingambiguities can occur in certain regions as a result of similaror repetitive texture patterns


(a) (b)

(d)

(c)

Target pixel

Subw

indow

Figure 4 Cost aggregation windows (a) 5times 5 pixel square window(b) adaptive window (c) windowwith adaptive support weights and(d) all six possible resulting shapes of adaptive windows

32 Cost Aggregation Cost aggregation is the most impor-tant stage for determining the general performance of astereo vision disparity map algorithm especially for localmethods The purpose of cost aggregation is to minimizematching uncertainties Cost aggregation is needed becausethe information obtained for a single pixel upon calculatingthe matching cost is not sufficient for precise matching Localmethods aggregate the matching cost by summing themover a support region [11] This support region is typicallydefined by a square window centered on the current pixel ofinterest as shown in Figure 4(a) The most straightforwardaggregation method is to apply a simple low-pass filter inthe square support window The fixed-size window (FW)technique (eg binomial or Gaussian uniform (box filters))suffers an increased error rate when the size of the supportwindow is increased over a certain threshold Moreover thismethod requires the parameters to be set to values suitablefor the particular input dataset Otherwise it tends to blurobject boundaries [49] To avoid fattening artifacts near depthdiscontinuities methods using shifting window or multiplewindows (MW) as well as methods using adaptive windows(AW) windows with adaptive sizes or adaptive supportweights (ASW) have been developed

In the MW technique multiple windows are selectedfrom among a number of candidates based on the supportwindows that produce smaller matching costs This methodwas implemented by Hirschmuller et al [50] and Veksler [51]in their previous studies of real time stereo vision disparitymap algorithms However their experimental results revealdifficulties in preserving dedicated pixel arrangements indisparity maps especially at object boundaries This occursbecause of the shape of the support windows This approachis imperfect for a small number of candidates To resolvethis problem the AW technique was developed to reduce theerrors in the disparity map caused by boundary problems Inthis method the support regions are constructed as approx-imations to the local image structures Figure 4(b) illustrates

the application of this method with five subwindows withdimensions of 3 times 3 These subwindows must be located nearthe target pixel as shown in Figure 4 The cost aggregationwith the minimum matching cost value for this pixel iscalculated For example the cost can be calculated as the sum-mation over the target pixel subwindow and any two otheradjacent subwindowsThe chosen shape of the validmatchingwindows for aggregation can therefore be any of the shapesshown in Figure 4(d) In practice the shape of the adaptivewindow is adaptively varied to reflect the local image contentsuch as corners and edges

TheAW technique was implemented by Lu et al [52] whoachieved high quality results both near depth discontinuitiesand in homogenous regions Lursquos work was improved uponby Zhang et al [53] through a modification to the conceptof adaptive support regions They developed support regionswith arbitrarily adaptive shapes and implemented the algo-rithmon aGPU for real time applicationsThe shapes of thesesupport regions are more flexible and are not restricted to berectangles These authors achieved high matching accuracywith real time implementation In this AW technique thealgorithm attempts to find support windows that fit the shapeor size of each region while preventing them from crossingobject boundaries Furthermore this technique is able toreduce computational costs as discussed by Chen and Su[54] These authors proposed a shape adaptive low com-plexity technique for eliminating computational redundancybetween stereo image pairs for pixelsmatchingThey groupedpixels with the same depth value to reduce the number ofcomputations

A comparative study of the use of different support regiontechniques in the cost aggregation stage was performed byFang et al [55] This study addressed the FW AW and ASWapproaches The authors concluded that the most advanta-geous technique for cost aggregation is the ASW approachIn this technique each pixel in the support region is assigneda support weight which depends on its intensity dissimilarityand spatial distance from the anchor pixel as shown inFigure 4(c) The target pixel which is located at the center isassigned different weight depending on distance as indicatedby the different tone of colors Generally for typical ASWtechniques (11) is used to aggregate the matching costs1198621015840(119901 119889) at pixel 119901 and disparity 119889

1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)

where 119908119901 is a square support window centered on pixel 119901The window size is a user defined parameter The value ofthe119882(119901 119902) function represents the possibility that a pixel 119902will possess a disparity value similar to that of the windowrsquoscenter pixel 119901 119862(119902 119889) represents a target pixel 119902 with adisparity value 119889 Ideally119882(119901 119902) should return a value of ldquo1rdquoif pixels119901 and 119902have equal disparity values and ldquo0rdquo otherwiseChen et al [56] developed a trilateral filter based on theASW approach with using a bilateral filter They also added anew weighted term to increase the robustness against objectboundaries


Essentially in ASW application a higher weight will beallocated to a pixel if its intensity is more similar to thatof the anchor pixel and if it is located at a smaller distancefrom the anchor pixel as implemented by Zhang et al [57]This method is able to produce a disparity map in which theobject boundaries are well preserved and the accuracy is veryhigh compared with the previous methods reported in theirliterature Hosni et al [58] presented an extensive evaluationof ASW regions They performed their test on a GPU toevaluate whether the speed and computational efficiencywere sufficient for real time responsesTheir evaluations indi-cated that the ASW approach produces outstanding resultsin terms of both computational efficiency and the qualityof the generated disparity maps Nalpantidis and Gasteratos[59] developed a new approach based on the ASW techniqueThey combined it with the quantified gestalt law to calculatea weighting factor In general a correlation weight reflectsthe proximity similarity and continuity between both inputimages (ie left and right images)

33 Disparity Computation and Optimization Generally astereo matching algorithm represents one of the two majoroptimizations approaches the local approach or the globalapproach In the local approach when the final disparities arecomputed the disparity for each pixel is essentially selectedusing a local winner takes all (WTA) strategy as define by

119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)

The disparity associated with the minimum aggregated cost119889119901 at each pixel is chosen 1198621015840(119901 119889) represents the aggregatecost obtained after the matching cost calculation and 119863denotes the set of all allowed discrete disparities The WTAstrategy is utilized in this stage in local algorithms such asthose implemented by Cigla and Alatan [46] Zhang et al[53] and Lee et al [60] According to their findings thedisparity maps obtained at this stage still contain errors inthe form of unmatched pixels or occluded regions Becausethe aggregation in local methods is performed throughsummation or averaging over support regions their accu-racy is sensitive to noise and unclear regions This occursbecause only local information from a small number of pixelssurrounding the pixel of interest is utilized to make eachdecision Therefore the accuracy of a local method at thisstage depends on the matching cost computation and costaggregation stages Subsequently in the disparity refinementstage the errors will be reduced using several filteringtechniques

By contrast in a global approach certain assumptionsare made about the depth of field of the scene which areusually expressed in an energyminimization frameworkThebulk of the effort in a global method is expended during thedisparity computation phase and the aggregation step is oftenskipped [11]Themost commonly adopted assumption is thatthe scene is locally smooth except for object boundaries andthus neighboring pixels should have very similar disparitiesThis constraint is referred to as a smoothness constraintin the stereo vision literature In the typical global stereo

vision formulation the objective is to find an optimal energydisparity assignment function 119889 = 119889(119909 119910) that minimizes

119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)

where 119864data(119889) represents the matching costs at the coordi-nates (119909 119910) the smoothness energy 119864smooth(119889) encouragesneighboring pixels to have similar disparities based on theprevious stated assumptions and 120573 is a weighting factor

A global method such as the belief propagation (BP)approach requires large amounts of computational resourcesand memory for the storage of the image data and theexecution of the algorithm For the improvement to theBP technique achieved implemented by Liang et al [61]which was implemented on the GPU the time required forprocessing is still large compared with that required by thelocal method strategy Wang et al [62] implemented globalapproach using a graph cut (GC) algorithm to optimize theenergy functionTheir method selects disparity values with alower energy value Another well-known global technique isa dynamic programming (DP) DP is executed for each scanline (row) independently resulting in polynomial complexityThe assumption adopted DP is that of an ordering constraintbetween neighboring pixels of the same row Recently themultiresolution energy minimization framework introducedby Arranz et al [14] achieved real time performance whilemaintaining the resolution of producing disparity maps Theadvantage of this framework is the reduction in computa-tional complexity that is achieved through the multiresolu-tion technique However for images of higher resolution andwith many more different levels of disparity the frameworkis unable to perform at a real time frame rate (30 frames persecond)

34 Disparity Map Refinement The purpose of the disparityrefinement stage is to reduce noise and improve the disparitymaps Typically the refinement step consists of regularizationand occlusion filling or interpolation The regularizationprocess will reduce the overall noise through the filteringof inconsistent pixels and small variations among pixels ondisparitymapTheocclusion filling or interpolation process isresponsible for approximating the disparity values in areas inwhich the disparity is unclear Typically occluded regions arefilled with disparities similar to those of the background ortextureless areas Usually the occlusion regions are detectedby using left-right consistency check such as those imple-mented by Yang et al [63] andHeo et al [64] If the matchingalgorithm rejects disparities with low confidence then theinterpolation algorithm will estimate approximations to thecorrect disparities based on the local neighborhoodsThe dis-parity refinement step normally combines local informationfrom the local neighborhood near each measurement with aconfidence metric

Two classic and common techniques for local disparityrefinement are Gaussian convolution and the median filterIn Gaussian convolution disparities are estimated in com-bination with those of neighboring pixels in accordancewith weights defined by a Gaussian distribution The pri-mary purpose of this method is to reduce the noise in


CPU

Libraries(OpenCV OpenGL

OpenCL etc)

(a)

CPU

Libraries

GPU

(CUDA OpenCVOpenGL OpenCL etc)

(b)

CPU

FPGA

VHDLVerilog etc

(c)

Figure 5 Block diagrams of the implementation of stereo vision disparity map algorithms (a) Software-based one (eg only a computer ora laptop) (b) hardware-based onewith a GPU implementation (eg a computer or a laptop built with a GPU unit) and (c) hardware-basedone with FPGA implementation connected to a computer to burn-in the codes and become a standalone system

the disparity map but the Gaussian filter also reduces theamount of fine detail present in the final disparity mapA technique developed by Vijayanagar et al [65] uses theweights defined by aGaussian filter to improve disparitymapsby approximating missing disparity values based nearby highconfidence disparity pixels as a guide to prevent filteringacross object boundaries Meanwhile themedian filter is ableto remove small isolated mismatches in disparity by virtueof its edge preserving property and it is suitable for real timeimplementation because of its low computational complexityThis filter selects the median value within window of pixelsas the final result for the central pixel In a study by Michaelet al [66] a disparitymap refinement approach usingmedianfilteringwas developed for a real time stereo vision algorithmFurthermore the median filter was modified by Ma et al[67] using the constant time weighted technique Theirmodification achieves high accuracy in removing noise anderror while maintaining the edges in disparity maps

The diffusion technique performs a function similar tothat of Gaussian convolutionMoreover there exists an adap-tation of this approach called anisotropic diffusion UnlikeGaussian convolution which destroys edges and fine detailsanisotropic diffusion applies smoothing without crossing anyedges as implemented by Banno and Ikeuchi [68] in the dis-parity map refinement stage of their disparity map algorithmTheir approach was improved upon by Vijayanagar et al [65]yielding a method called multiresolution anisotropic diffu-sion In thismethod the disparitymap is downsampled usingthree different resolution factors At each resolution 35 itera-tions of the anisotropic diffusion process are performed Theresult of the proposed algorithm is free of occlusion errorsand the edges in the disparitymap have been refined Anotherapproach which was developed by Zhang et al [6] employsa two-step process to further refine the estimated disparitymapThe authors presented the results they achieved througha color image guided depth matting process in a frameworkbased on Bayesian matting and 2D polynomial regressionsmoothing techniques This technique was found to be usedto be effectively preserving the discontinuities at objectboundaries while achieving smoothing in flat regions

4 Software-Based Stereo VisionDisparity Map Algorithms

This section reviews several software-based implementationprocesses of global and local methods for the generation ofdisparity maps These algorithms were developed and testedusing only a CPU as the processing hardware as shown inFigure 5 A software-based implementation is designed to usethe CPU to interface with API software The API softwareprovides a set of libraries such as Open Computer Vision(OpenCV) Open Computing Language (OpenCL) librariesand Open Graphic Language (OpenGL) A previous sum-mary of software-based stereo vision disparitymap algorithmand their performances was presented by Brown et al [19]The discussion also addressed the corresponding methodsand occlusion handling techniques In essence the differ-ences in these algorithms lie in the cost aggregation stage andin the optimization of the building blocks which determinethe main characteristics of the developed algorithms Severalresearchers such as Park et al [69] and Cigla et al [70] havedeveloped new algorithms by taking advantages of local andglobal methods for handling occlusions object boundariesand untextured regions These techniques follow an iterativeprocess for allocating disparities that spread into certain seg-ments by applying pixel similarity constraints consideringoverlapping regions enforcing smoothness between similarlycolored neighboring segments and penalizing occlusionsSeveral other researchers have applied other global tech-niques with various modifications resulting in a semiglobalapproachThismethod involves dynamic programming opti-mization such as that implemented by Hirschmuller [71]and Salmen et al [16] The results reported by these authorsindicate low computational complexity

41 Global Approaches The best known approaches amongthe global methods are belief propagation (BP) and graphcut (GC) algorithms Perez and Sanchez [17] used the BPapproach to develop a real time high-definition algorithmthat outperformed classical BP by implementing two BP algo-rithms in their 3D telepresence systems The first instance of


BP performs a classification of the pixels into areas designatedas reliable containing occlusion errors and textureless toreduce the numbers of memory accesses required for thesethree groups of pixels The second BP process is used todecreasememory traffic by generating the final disparity mapwith a reduced number of iterations due to information fromprevious BP iterations The experimental results demon-strated improved performance The authors compared thisapproach with classical BP and observed a 90 improvementin efficiency Wang and Yang [72] implemented groundcontrol points (GCP) influenced by the MRF model Intheir method GCP-based regularization for the optimizationframework is performed using a Bayesian rule Meanwhilethe energy minimization technique for finding an optimalsolution to the inference problem is implemented using theGC approach Evaluations of this method demonstrated itseffectiveness at improving the disparity map reconstructionto regularize problems of incorrect stereo matching Theapproaches developed by Perez and Sanchez [17] and WangandYang [72] both produce accurate disparitymaps but theircomputational costs remain high However Wang et al [18]proposed a hierarchical bilateral disparity structure (HBDS)algorithm based on a GC technique to reduce computa-tional complexity and improve the accuracy of the generateddisparity maps These authors divide all disparity levelshierarchically into a series of bilateral disparity structuresto increase the fineness of the disparity map During therefinement stage any fattened regions are recalibrated basedon the disparity values of all nearby pixels The evaluationresults indicated good performance with reduced processingtime and improved disparity map accuracy

A new technique was proposed by Chen and Lai [73]based on augmenting paths and the adoption of a push-relabelling scheme The augmenting path algorithm func-tions by using multiple threads to calculate each block indi-vidually and to select another completed nearby block withwhich to merge The proposed method identifies the inde-pendent processing loops in theGC approach and isolates thecomputation of each loop Each image is sliced into smallerimage segments which will then be processed in parallelThe proposed method enables a remarkable decrease in exe-cution time by a factor of 47 compared with the originalGC approach but considerable computational programmingefforts are required Kolmogorov et al [74] developed fourdifferent smoothness terms (ie data smoothness occlusionand uniqueness) to improve the accuracy of their resultsTheobjectives of theirmethod are to reduce the errors in occludedareas and increase the efficiency during postprocessingWanget al [75] developed an algorithm using the MRF frameworkto eliminate holes and misaligned pixels Their work pro-duced high quality disparity maps but also required complexcomputational programming Ploumpis et al [76] developeda new stereo matching approach based on particle filters andscattered control landmarks The proposed method consistsof three steps First multiple disparity maps are used toacquire a set of features or landmarks and then segment theimages Afterward to estimate the best disparity values scanline particle filtering is applied In the last step a Markovchain model is employed to reduce the computational

redundancy of the particle filtering process Using thismethod high quality disparity maps can be produced

42 Local Approaches In local methods pixel correspon-dences are generated by measuring the correspondenceand similarity between image regions and very effectiveimplementation can be produced using this approach [77]The assignment of disparity values is achieved by applyingWTA strategy after calculating each candidate disparity valueindividually The matching cost function is aggregated viaa summation or an averaging over a support region Thedisparity value with the minimum cost for each pixel isassigned to that pixel An algorithm based on an efficient costaggregation strategy was proposed by Mattoccia et al [13]These authors used joint bilateral filtering and expanded thecalculation structures that allow for the efficient and accurategeneration of disparity maps The idea behind adoptingthe selected bilateral filtering approach in the developedalgorithm is to combine a geometric constraint (ie a spatialfilter) with color proximity constraint (ie a range filter)The performance of the proposed approach was tested fornoise and accuracy using the Middlebury dataset Anotherrefinement technique was proposed by Psota et al [78] whichdoes not use image segmentation or plane fitting Insteadthe algorithm performs iterative refinement of the resultsof adaptive weight stereo matching In each iteration ofdisparity refinement the algorithm uses the ASW approachto penalize disparity differences in local windows A total ofeight iterations on the Middlebury dataset were performedby Psota et al and the correspondence error percentage wasobserved to decrease from 146 to 083 A new techniquefor local cost aggregation for stereo matching was proposedby Yang [79] In this technique the matching cost values areaggregated adaptively based on a tree structure The nodesof this tree consist of all of the image pixels and the treecontains all edges between nearest neighboring pixels Aspanning tree can be computed by removing unwanted edgesEdge with high weights will be removed during spanningtree construction Then the minimum spanning tree MSTapproach is applied to obtain the sum of the minimumvalues of all spanning trees Yangrsquos method offers a lowcomputational complexity and high accuracy but has notbeen tested for use in real time implementation

Xu et al [80] proposed an algorithm that calculates theaggregation cost via the join optimization of both the leftand right matching costs The authors assign reasonableweighting coefficient and exclude occluded pixels whilepreserving sufficient support windows for accuratematchingThe result is the ability to reduce unwanted pixels in theforeground and increase accuracy in highly textured regionsFurthermore Lee et al [60] developed an algorithm basedon local approach with no iteration using three-mode crossCT with noise buffering to increase the robustness againstimage noise in textureless areas This technique also providestwo bits of cross CTwithin threemodes of implementation toincrease the reliability of the census measure Most disparitymethods encounter difficulties when confronted with fast-moving objects but Leersquos algorithm addresses this problemby using the concept of optical flow to support weight


Table 3 Previous evaluation papers on CPU FPGA and GPUYear Author Focus

2008 Gac et al [41]

Presented a performance evaluation of different target architectures which are FPGA (XilinxVirtex 2 Pro) GPU (Nvidia Geforce 8880) and CPU (Xeon dual core 3GHz) performance basedon back projection technique with several ways to speed up They have reduced the computationaltime for these three different architectures through memory parallelized architecture

2010 Kalarot and Morris[87]

Reviewed performance on FPGA (Altera Stratix III) and GPU (Nvidia Geforce GTX 280)performance based on strengths and limitations The same DP algorithm has been applied toFPGA and GPUThe evaluations have been made on internal clocks memory space and disparityrange of 128 and 256

2012 Pauwels et al [88]Reviewed performance on FPGA (Xilinx Virtex 4) and GPU (Nvidia Geforce GTX 7900)performance based on optical flow stereo vision and local image features including energyorientation and phase

2012 Russo et al [90]Presented performance comparison on FPGA (cyclone II) and GPU (Nvidia Geforce GTX 295)for image convolution processing The performance is measured through clock cycle ratio andexecution time

2013 Fowers et al [91]

Presented performance comparison on FPGA (Altera Stratix III) GPU (Nvidia Geforce GTX 295)and CPU (Xeon Quad Core W3520) for sliding window applications They have used SADalgorithm for all three architectures The performance comparison is based on energy efficiencyand time consuming for image processing Multiple windows sizes (4 times 4 9 times 9 16 times 16 25 times 25and 45 times 45) have been used to evaluate the best performance architecture

2014 Xu et al [89] Presented the speed performance of CPU (AMD Opteron processor 6366HE Intel Xeonprocessor E5-2620) and GPU (Nvidia GeForce GTX 770) on the pyramidal stereo algorithm

computations within a localized window Koo et al [81]used a gradient based matching technique to reduce theradiometric errors and improve the matching cost functionby using a Gaussian-based weighting functionThe referenceimage is divided into two difference images corresponding tolow and high frequencies Then the Difference of Gaussian(DoG) function is employed to reduce the errors that ariseduring the matching process The authors demonstrated areduction of errors on a sample set of images acquired inan outdoor environment Matsuo et al [82] used a localapproach based on the AD algorithm and the Sobel operatorin the matching cost calculation stage and box filtering withthe WTA optimization in the cost aggregation stage Theyused a weighted joint bilateral filter (JBF) in the refinementstage They produced accurate disparity maps using severaliterations and a fixedwindow size for the JBFNalpantidis andGasteratos [83] developed a new stereo matching algorithmthat employs the AD algorithm and performs aggregationby considering the gestalt laws of proximity similarity andcontinuity within a psychophysically based weight assign-ment framework Their proposed algorithm yielded accurateresults when applied to the Middlebury dataset

5 Real Time Stereo Vision Disparity MapAlgorithms Using Additional Hardware

The ability to implement stereo matching algorithms in realtime represents a new research area in the field of computervision The results of the online Middlebury benchmarkingsystem established by Scharstein and Szeliski [35] indi-cate that algorithms developed using a parallel processingapproach or additional hardware are able to deliver pro-cessing times among the best ones achieved on a standardbenchmarking dataset Real time stereo vision algorithms

are able to achieve rates of greater than 30 frames persecond in their disparity mapping output In this section thediscussion is limited to platforms that use FPGAs and GPUsfor real time implementation of stereo vision algorithmsFigure 5 shows a basic block diagram for such hardware-based implementation on FPGA or GPUThe FPGA consistsof a CPU a multi-inputoutput port and a large set ofconfigurable logic blocks (CLB) that can be configuredaccording to the developerrsquos preferred design The featuresof FPGA include versatility and the flexibility to operateeither as standalone systems or as coprocessors on expansioncards for computer FPGAs are most often programmedusing hardware description language (HDL) [84] and Baconet al [85] By contrast a GPU is a dedicated coprocessorwith a fixed architecture that enables the acceleration of therendering of 2D and 3D graphics by offloading the relatedprocesses from the CPU Recent GPU designs have evolvedfrom being dedicated graphics rendering processors to moregeneral parallel processors A GPU consists of multipleprocessors with more than hundred cores depending on themodel A GPU is able to operate in combination with CPUand open-source libraries such as the OpenCL OpenCV andComputeUnifiedDeviceArchitecture (CUDA) libraries [86]

51 Comparative Studies of FPGAs and GPUs Several papersrelated to performance evaluations of FPGAs GPUs andCPUs were reviewed for this literature survey as specified inTable 3 An early comparison among the FPGA GPU andCPU was conducted by Gac et al [41] The results indicatedthat the GPU achieved the highest absolute performancein terms of reconstruction time The authors applied theback projection technique in their global algorithm via 3Dtomography image reconstruction Kalarot and Morris [87]compared the performance of theDP algorithm implemented


on the FPGA and GPU when applied to their own rectifiedimages for different disparity range Their results indicatedthat the FPGA offered faster processing than the GPU for adisparity range below 128 but that the FPGA was unable tohandle a disparity range of greater than 256 unlike the GPUThis finding can be attributed to the memory limitationsof standalone FPGA systems which prevent their use forprocessing large images Another structured evaluation andcomparison of the FPGAandGPUwas performedbyPauwelset al [88] in the context of a real time analysis of opticalflow local image features and stereo vision applications Theauthors applied their method to the Middlebury dataset Thecomparison was performed based on the hardware architec-ture speed data dependency accuracy and time required todesign the structure of the algorithms The presented resultsdemonstrated that the GPU implementation was superior inall respects and yieldedmore accurate and faster results whenimplemented as real time stereo vision systems comparedwith the FPGA implementation Xu et al [89] compared thespeed performances achieved using CPU and a GPU for theirpyramidal stereo algorithm Their results indicated that theGPU was 12x faster than the CPU

Russo et al [90] performed a performance comparisonof FPGA and a GPU for image convolution processing Theyreported that the GPU exhibited the better performance interms of execution time and number of clock cycles Thisfinding was attributed to the fact that the GPU tended to bebetter able to explore the extremely large amounts of datacontained within the high-resolution images Moreover thecharacteristic features of a GPU such as multiple pipelinesand high bandwidth assist in enhancing its performanceFowers et al [91] conducted a performance and energycomparison of a GPU FPGA and a CPU with a multicorearchitecture They used the SAD algorithm with the slidingwindow technique as the algorithm implemented for thecomparison It was found that the FPGA provided the bestenergy efficiency whereas the GPU delivered the best per-formance Jin andMaruyama [92] compared implementationof their algorithm on the FPGA and a GPU based on speedand accuracy They also proposed a method of improv-ing the circuit design for FPGAs to reduce the requiredmemory resources while maintaining accuracy However theresults reported in all of the reviewed papers are subjectto processing performance which depend on the availablehardware resources and the computational requirements ofthe considered task A recent study of FPGAs was conductedby Lentaris et al [93] for their ongoing projects SPARTANSEXTANT and COMPASS to improve the behaviour ofautonomous planetary exploration rovers The study focusedon the potential use of FPGAs for implementing a varietyof stereo correspondence feature extraction and visualodometry algorithms

52 Global Approaches Several global methods have beenimplemented on FPGAs and GPUs for the developmentof real time stereo vision disparity map algorithms Aglobal optimization algorithm for stereo matching based onimprovement to the BP approach implemented on a GPUwas presented by Xiang et al [9] Their technique involves

the integration of color-weighted correlations to improvehierarchical BP Occlusion problems are resolved by combin-ing a uniqueness constraint and a similarity constraint forthe detection of occluded regions The approach of Xiang etal outperforms other BP methods with regard to their realtime implementation on GPUs However its results in dis-continuous regions of the disparity maps are somewhat poorand it requires more complex computations The approachalso suffers an increased time delay when the algorithm isattempting to generate accurate results for such discontin-uous regions An improvement to the poor quality of thedisparity maps was achieved by Wang et al [94] They usedthe AD-CT algorithm in the matching cost calculation stagewith a semiglobal optimization framework on FPGA boardSemiglobal optimization involves optimizing the smoothnessof the disparity map along different directions separatelyThe designated directions are along lines traveling to theright bottom bottom right and bottom left This systemwas found to be able to adjust the image resolution anddegree of parallelism to achieve maximum efficiency Theresult was the ability to produce high quality disparity mapsfrom high-definition images In [95] a new cost functionwas developed for the matching of corresponding pixels Theauthors proposed a parallel approach to a variant of a globalmatching cost calculationmethod implemented on aGPU forsymmetric stereo images A bank of log-Gabor wavelets wasdeveloped for the analysis of such symmetric images in thespectral domain Using a GPU the authors achieved real timedisparity estimations for high-resolution images

Implementation of a new algorithmusing theGCmethodon a GPU was presented by Choi and Park [96] Theybuilt their algorithm to operate in three stages using thegraph construction method to accelerate the convergence ofthe GC calculation A reordering heuristic and initializationmethod were employed to further increase the executionspeed based on the proposed graph construction methodThen a repetitive block based push and relabel method wasused to increase the data transfer efficiency Finally theyused low-overhead global relabelling algorithm to increasethe GPU occupancy They achieved an improved executiontime compared with typical global methods at the cost ofconsiderable programming efforts Yao et al [97] treatedimage warping as an energy minimization problem Firstthey developed a sparse disparity map by means of stereomatching process Then the map was warped an energyminimization function with three independent terms (ie adisparity constraint a structure constraint and a temporalconstraint) They applied their method to test images withdifferent resolutions and evaluated the results that are basedon the execution times required for the GPU and CPUimplementation The GPU runtime was 24x faster than theCPUprocessing time satisfying the requirements of real timeoperation

53 Local Approach Current FPGA technology offers thou-sands of small logic blocks embedded in the connectionmatrix This allows arbitrary computation blocks to be con-structed frombasic computing blocks through parallel circuitconnections Detailed summary of information regarding


the advantages and disadvantages of real time implementa-tion of stereo vision algorithms on FPGAs has been pro-vided by Samarawickrama [98] Kalarot and Morris [87]implemented an algorithm on FPGA using a fast and simpleapproach by combining the distortion removal and alignmentcorrection tasks in a single step by means of lookup tablesHowever a problem was encountered in the case of images ofmore than 1 megapixel in size at which the FPGAwas unableto process because of its very limited onboard memory Thismemory limitation makes FPGAs unsuitable for the process-ing of high-definition images unless external memory is usedto support itMattoccia [99] performed a comparison of threedifferent algorithms namely a fixed window algorithm anASW algorithm and a semiglobal algorithm implementedon FPGAs Mattocciarsquos results demonstrated that the outputdisparity maps were fairly accurate for all tested algorithmsRecently Colodro-Conde et al [100] implemented an areabased stereo matching algorithm on FPGA board and testedits performance using the Middlebury dataset They devel-oped the algorithm to use the SAD approach for thematchingcost calculation and the median filter to refine the disparitymaps Their architecture design involved multiple buffers fortemporary memory storage In this design when the windowsize is increased the buffers also need to be increased forthe parallel processing of the allocated memory Howeverthe memory size and inherent frequency of FPGA limitits suitability for such tasks and applications especially forreal time applications Nalpantidis et al [20] used FPGAto prepare an efficient implementation of their hierarchicalmatching algorithm on uncalibrated stereo vision images Intheir approach two-dimensional correspondence search isperformed using a hierarchical technique Then the inter-mediate results are refined by three-dimensional cellularautomata (CA)The final disparity value is defined in terms ofthe distance between the matching positions This proposedalgorithm is able to process uncalibrated and nonrectifiedstereo images when implemented on the FPGA

Excessive time consumption is the main challenge fac-ing real time algorithm implementation because of theircomputational complexity The reasons that real time visionalgorithms are generally suitable for implementation onGPUs have been explained and discussed by Kim et al [107]A GPU unit is able to run the same instructions on multiplesets of data simultaneously Based on the functionalityMei et al [45] developed a stereo matching algorithm forimplementation on a GPU with good performance in termsof both accuracy and speed The matching cost value wasinitialized using the AD measure and CT The cost wasaggregated in dynamic cross based regions and updated ina multidirection scan line optimization Several researchershave developed algorithms based on domain transforma-tions This technique was previously initiated by Gastal andOliveira [108] who used a transformation technique thatenables the aggregation of 2D cost data using a sequence of 1Dfilters This technique was improved upon by Pham and Jeon[24] by means of dimensionality reduction technique Theadvantage of this technique is that it reduces the complexityof the computational requirements compared with a 2Dcost aggregation calculation A multiresolution anisotropic

diffusion approach based on a disparity refinement algorithmthat can be executed in a real time environment was proposedby Vijayanagar et al [65] This algorithm exploits the imagepyramid concept to gradually enhance the disparity mapat different levels of resolution and to align the objectboundaries in color imagesThis technique allows smoothingto be achieved without loss of edges making it a useful toolfor improving image segmentation

A novel local method for stereo matching using a GPUwas presented byKowalczuk et al [104]The algorithmbeginswith an approximation based on ASW aggregation and alow-complexity iterative disparity refinement technique Theprobabilistic framework combines the summation term intoa matching cost minimization via a series of approximationsand facilitates interactive processing to improve the accuracyof the disparity map The refinement algorithm operatesby calculating the estimated disparity value of each pixelduring the current iteration using nearby pixel disparitiesfrom previous iterationsThe implementation of this methodof cost aggregation and iterative disparity refinement wasperformed by Yoon and Kweon [102] Instead of searchingfor the matching window with the optimal size and shapeit is possible to aggregate costs after local smoothing withina corresponding window to reduce matching noise Usuallynoise can be effectively reduced by applying a linear filter suchas a Gaussian filter but the resulting disparity map alwaysexhibits edge fattening Therefore to address this problem ofmismatching pixels or noise around regions of discontinuitydisparity maps Lin et al [105] proposed a new algorithmbased on an edge preserving filter for the ASW methodcomputed using a hierarchical clustering algorithm Thisalgorithm used a novel cost aggregation block to computecorresponding response for all the corresponding pixels in aset of sampling points Tippetts et al [106] used an intensityprofile shape-matching algorithm implemented on an FPGAto achieve real time estimations for microscale unmannedvehicles (ie helicopter) The algorithm consists of threesteps In the first stage filtering is performed using aGaussiankernel Then the shapes of target objects are identified on arow-by-row basis In the last stage after the entire image hasbeen processed a vertical smoothing filter is applied to reducethe remaining noise The authors also presented designs forFPGA blocks for each stage of implementation

54 The Challenges of Implementing Stereo Vision Algorithmson GPUs and FPGAs Over the past decade developmentsin computing architectures have exhibited a clear trendtoward increased heterogeneity and parallelism with mostmainstream microprocessors now possessing multiple coresand robust system architectures [109] At the same time theincreasing number of accelerator options has considerablyincreased the complexity of application design because ofthe need to perform an extensive exploration of the availabledesign space when attempting to choose a suitable deviceAlthough GPUs with CUDA have come into common use asaccelerators because of their low cost ready availability andsimple programming model comparable to that of FPGAsEkstrand et al [110] Perez-Patricio et al [111] Stein [112]and Long et al [113] have all presented results that different


Left image Right image Ground-truth image

Tsukuba

Venus

Teddy

Cones

Figure 6 Standard benchmarking images provided in the Middlebury Stereo Vision datasets in [35]

devices are better suited for different applications Thereforesufficient exploration of the different available devices fordifferent applications is critical to prevent researchers fromselecting unsuitable devices during the design phase Inthis survey for the summary of performance comparisonsbetween GPUs and FPGAs the results vary among differ-ent implementation and application domains None of theplatforms appears to be universally superior The preferreddesign depends on the specifications of the target platformHowever the use of GPUs and FPGAs can facilitate increased

speeds and reduced execution times The challenge for a newresearcher in this field is to determine how to develop analgorithm that is appropriate to a specific application and themost suitable platform

6 Accuracy Measurement

There are several academic research centers that providequalitative accuracy assessments of disparity maps throughonline submissions The datasets used for these assessments


PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)

Figure 7The performance of local algorithms in terms of bad pixelpercentages for the nonocc all and disc which are cited from [35]

can also be downloaded from the associated web pagesfor example the Middlebury Computer Vision pages [35]KITTI Vision Benchmark [114] and DIBRIS [115] Thesedatasets include both static and dynamic scenes Most of thearticles reviewed in this survey use the qualitative accuracymeasurements provided by [35] Thus this paper uses thesame resource to report the accuracy performance of a stereovision algorithm in terms of the percentage of bad pixelsAccording to Scharstein and Szeliski [11] the evaluation of theaccuracy level for each image is based on three attributes (iethe percentages of bad pixels among all pixels in nonoccludedregions (nonocc) all pixels detected as valid pixels (all) andpixels in regions near depth discontinuities and occludedregions (disc)) Four standard benchmarking images are usedin this evaluation these images are Tsukuba Venus Teddyand Cones and the original images and ground-truth imagesfor each are shown in Figure 6 Figures 7 and 8 show thenonocc all and disc results obtained on these images forapproximately 60 algorithms selected based on both local andglobal optimizations methods The bad pixel percentages ofthese algorithms are among the lowest values represented inthe database of [35]

Because this section discusses only accuracy measure-ments cited from [35] which are based on online submissionsthe implementation of the algorithms in Figures 7 and 8is not specified as either software-based or hardware-basedThere is some possibility that accuracy improvement can beachieved through implementation on additional hardwareas shown by Pauwels et al [88] However as reported byKalarot andMorris [87] the primary advantage of hardware-based (ie FPGA andGPU) implementation is that the speedor execution time can be tremendously improved comparedwith implementation using only a CPU Figure 9 showsthe average errors of local and global methods Here thealgorithms are represented by numbers which correspondto the algorithms represented at the same 119909-axis positionsin Figure 7 (ie for local methods) and 8 (ie for globalmethods) This figure shows only the accuracy performancesof existingmethods and is intended as a guidance or referencefor those who wish to develop their own algorithms

Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)

Figure 8 The performance of global algorithms in terms of badpixel percentages for the nonocc all and disc which are cited from[35]

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm

Total averageLocalGlobal

000100200300400500600700800

Bad

pixe

l (

)

Figure 9 The average errors in terms of bad pixel percentage forlocal and global methods

7 Conclusion

The stereo matching problem remains a challenge for com-puter vision researchers A literature survey of the latest stereovision disparity map algorithms is provided here and all citedalgorithms are categorized according to the processing stepswith which they are associated in the taxonomy of Scharsteinand Szeliski Becoming familiar with the state-of-the-artalgorithms for stereo vision disparity mapping is a timeconsuming task In this survey of the latest developmentsin the area of stereo matching algorithms the processingsteps composing such an algorithm and their software-basedas well as hardware-based implementation was thereforeperformed and presented to assist in this taskThe qualitativemeasurement of the accuracy of such algorithmswas also dis-cussed To assist the reader in navigating the numerous workspresented Table 2 is presented as a summary It specifies thesteps and computational platforms used in each approach asa reference for the development of new algorithms

Nomenclature

119868119897 Left image119868119903 Right image119889 Disparity value(119909 119910) Pixel coordinates


119908 Support windowisin Element of(119894 119895) Neighboring pixel coordinates119908119901 Support window centered on

pixel 119901119901 Pixel of interest119889119901 Smallest pixel value119863 Set of disparity values120573 Weighting factor119864 Energy function119902 Neighboring pixels in the

support window

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported in part by the Universiti SainsMalaysia Research University Individual (RUI) with Accountno 1001PELECT814169 and the Universiti Teknikal Malay-sia Melaka

References

[1] R Szeliski Computer Vision Algorithms and ApplicationsSpringer 2010

[2] B H Bodkin Real-time mobile stereo vision [MS thesis] Uni-versity of Tennessee 2012

[3] M Hansard S Lee O Choi and R P Horaud Time-of-FlightCameras Principles Methods and Applications Springer 2012

[4] S Foix G Alenya and C Torras ldquoLock-in time-of-flight (ToF)cameras a surveyrdquo IEEE Sensors Journal vol 11 no 9 pp 1917ndash1926 2011

[5] M Y Kim S M Ayaz J Park and Y Roh ldquoAdaptive 3D sensingsystem based on variable magnification using stereo vision andstructured lightrdquo Optics and Lasers in Engineering vol 55 pp113ndash127 2014

[6] S Zhang C Wang and S C Chan ldquoA new high resolutiondepth map estimation system using stereo vision and depthsensing devicerdquo in Proceedings of the IEEE 9th InternationalColloquium on Signal Processing and Its Applications (CSPA rsquo13)pp 49ndash53 IEEE Kuala Lumpur Malaysia March 2013

[7] W Kazmi S Foix G Alenya and H J Andersen ldquoIndoor andoutdoor depth imaging of leaves with time-of-flight and stereovision sensors analysis and comparisonrdquo ISPRS Journal ofPhotogrammetry and Remote Sensing vol 88 pp 128ndash146 2014

[8] B Tippetts D J Lee K Lillywhite and J Archibald ldquoReviewof stereo vision algorithms and their suitability for resource-limited systemsrdquo Journal of Real-Time Image Processing pp 1ndash212013

[9] X Xiang M Zhang G Li Y He and Z Pan ldquoReal-time stereomatching based on fast belief propagationrdquoMachine Vision andApplications vol 23 no 6 pp 1219ndash1227 2012

[10] L Nalpantidis and A Gasteratos ldquoStereovision-based fuzzyobstacle avoidance methodrdquo International Journal of HumanoidRobotics vol 8 no 1 pp 169ndash183 2011

[11] D Scharstein and R Szeliski ldquoA taxonomy and evaluation ofdense two-frame stereo correspondence algorithmsrdquo Interna-tional Journal of ComputerVision vol 47 no 1ndash3 pp 7ndash42 2002

[12] N Lazaros G C Sirakoulis and A Gasteratos ldquoReview ofstereo vision algorithms from software to hardwarerdquo Interna-tional Journal of Optomechatronics vol 2 no 4 pp 435ndash4622008

[13] S Mattoccia S Giardino and A Gambini ldquoAccurate andefficient cost aggregation strategy for stereo correspondencebased on approximated joint bilateral filteringrdquo in Proceedingsof the Asian Conference on Computer Vision (ACCV rsquo10) vol 38pp 371ndash380 Xirsquoan China September 2010

[14] A Arranz A Sanchez and M Alvar ldquoMultiresolution energyminimisation framework for stereo matchingrdquo IET ComputerVision vol 6 no 5 pp 425ndash434 2012

[15] L Xu O C Au W Sun et al ldquoStereo matching by adaptiveweighting selection based cost aggregationrdquo in Proceedings ofthe IEEE International Symposium on Circuits and Systems(ISCAS rsquo13) pp 1420ndash1423 Beijing China May 2013

[16] J Salmen M Schlipsing J Edelbrunner S Hegemann and SLuke ldquoReal-time stereo vision making more out of dynamicprogrammingrdquo inComputer Analysis of Images and Patterns pp1096ndash1103 Springer Berlin Germany 2009

[17] J M Perez and P Sanchez ldquoReal-time stereo matching usingmemory-efficient Belief Propagation for high-definition 3Dtelepresence systemsrdquo Pattern Recognition Letters vol 32 no16 pp 2250ndash2253 2011

[18] Y-C Wang C-P Tung and P-C Chung ldquoEfficient disparityestimation using hierarchical bilateral disparity structure basedgraph cut algorithm with a foreground boundary refinementmechanismrdquo IEEE Transactions on Circuits and Systems forVideo Technology vol 23 no 5 pp 784ndash801 2013

[19] M Z Brown D Burschka and G D Hager ldquoAdvances incomputational stereordquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 25 no 8 pp 993ndash1008 2003

[20] L Nalpantidis A Amanatiadis G C Sirakoulis and A Gaster-atos ldquoEfficient hierarchical matching algorithm for processinguncalibrated stereo vision images and its hardware architec-turerdquo IET Image Processing vol 5 no 5 pp 481ndash492 2011

[21] T Song B Tang M Zhao and L Deng ldquoAn accurate 3-D firelocation method based on sub-pixel edge detection and non-parametric stereo matchingrdquo Measurement vol 50 no 1 pp160ndash171 2014

[22] L Wang M Gong M Gong and R Yang ldquoHow far can we gowith local optimization in real-time stereo matchingrdquo in Pro-ceedings of the 3rd International Symposium on 3DData Process-ing Visualization and Transmission pp 129ndash136 Chapel HillNC USA June 2006

[23] D Min J Lu and M N Do ldquoA revisit to cost aggregation instereo matching how far can we reduce its computationalredundancyrdquo in Proceedings of the IEEE International Confer-ence on Computer Vision (ICCV rsquo11) pp 1567ndash1574 BarcelonaSpain November 2011

[24] C C Pham and J W Jeon ldquoDomain transformation-based effi-cient cost aggregation for local stereo matchingrdquo IEEE Transac-tions on Circuits and Systems for Video Technology vol 23 no 7pp 1119ndash1130 2013

[25] Q Yang R Yang J Davis and D Nister ldquoSpatial-depth superresolution for range imagesrdquo in Proceedings of the IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)pp 1ndash8 IEEE Minneapolis Minn USA June 2007


[26] A Miron S Ainouz A Rogozan and A Bensrhair ldquoA robustcost function for stereomatching of road scenesrdquoPattern Recog-nition Letters vol 38 no 1 pp 70ndash77 2014

[27] H Huang and Q Wang ldquoA region and feature-based matchingalgorithm for dynamic object recognitionrdquo in Proceedings ofthe IEEE International Conference on Intelligent Computing andIntelligent Systems (ICIS rsquo10) pp 735ndash739 Xiamen China Octo-ber 2010

[28] K Sharma K Jeong and S-G Kim ldquoVision based autonomousvehicle navigation with self-organizing map feature matchingtechniquerdquo in Proceedings of the International Conference onControl Automation and Systems pp 946ndash949 Gyeonggi-doRepublic of Korea October 2011

[29] L Wang Novel dense stereo algorithms for high-quality depthestimation from images [Theses and DissertationsmdashComputerScience] University of Kentucky 2012

[30] J Liu X Sang C Jia N Guo Y Liu andG Shi ldquoEfficient stereomatching algorithm with edge-detectingrdquo in OptoelectronicImaging andMultimedia Technology III vol 9273 of Proceedingsof SPIE p 7 November 2014

[31] F Ekstrand C Ahlberg M Ekstrom and G SpampinatoldquoHigh-speed segmentation-driven high-resolution matchingrdquoin Seventh International Conference on Machine Vision (ICMV2014) p 94451Y International Society forOptics and Photonics2015

[32] B J Tippetts D-J Lee J K Archibald and K D LillywhiteldquoDense disparity real-time stereo vision algorithm for resource-limited systemsrdquo IEEE Transactions on Circuits and Systems forVideo Technology vol 21 no 10 pp 1547ndash1555 2011

[33] S H Lee and S Sharma ldquoReal-time disparity estimation algo-rithm for stereo camera systemsrdquo IEEE Transactions on Con-sumer Electronics vol 57 no 3 pp 1018ndash1026 2011

[34] R K Gupta and S-Y Cho ldquoWindow-based approach for faststereo correspondencerdquo IET Computer Vision vol 7 no 2 pp123ndash134 2013

[35] D Scharstein and R Szeliski Middlebury Stereo EvaluationmdashVersion 2 2015

[36] A Fusiello U Castellani and V Murino ldquoRelaxing symmetricmultiple windows stereo using Markov Random Fieldsrdquo inComputer Vision and Pattern Recognition vol 2134 of LectureNotes in Computer Science pp 91ndash105 Springer 2001

[37] R Yang and M Pollefeys ldquoMulti-resolution real-time stereoon commodity graphics hardwarerdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition vol 1 pp I211ndashI217 Madison Wis USA June2003

[38] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereomatchingrdquo in Proceedings of the IEEEConferenceon Computer Vision and Pattern Recognition (CVPR rsquo07) pp 1ndash8 Minneapolis Minn USA June 2007

[39] S Satoh ldquoSimple low-dimensional features approximatingNCC-based image matchingrdquo Pattern Recognition Letters vol32 no 14 pp 1902ndash1911 2011

[40] F Cheng H Zhang D Yuan and M Sun ldquoStereo matching byusing the global edge constraintrdquo Neurocomputing vol 131 pp217ndash226 2014

[41] N Gac S Mancini M Desvignes and D Houzet ldquoHigh speed3D tomography on CPU GPU and FPGArdquo EURASIP Journalon Embedded Systems vol 2008 Article ID 930250 12 pages2008

[42] G Zhao Y K Du and Y D Tang ldquoA new extension of the ranktransform for stereo matchingrdquo Advanced Engineering Forumvol 2 pp 182ndash187 2011

[43] M Humenberger C Zinner M Weber W Kubinger and MVincze ldquoA fast stereo matching algorithm suitable for embed-ded real-time systemsrdquo Computer Vision and Image Under-standing vol 114 no 11 pp 1180ndash1202 2010

[44] L Ma J Li J Ma and H Zhang ldquoA modified census transformbased on the neighborhood information for stereo matchingalgorithmrdquo in Proceedings of the 7th International Conferenceon Image and Graphics (ICIG rsquo13) pp 533ndash538 IEEE QingdaoChina July 2013

[45] X Mei X Sun M Zhou S Jiao H Wang and X Zhang ldquoOnbuilding an accurate stereo matching system on graphics hard-warerdquo in Proceedings of the IEEE International Conference onComputer VisionWorkshops (ICCVWorkshops rsquo11) pp 467ndash474Barcelona Spain November 2011

[46] C Cigla and A A Alatan ldquoInformation permeability for stereomatchingrdquo Signal Processing Image Communication vol 28 no9 pp 1072ndash1088 2013

[47] N Zhang H Wang and J Cr ldquoA near real-time color stereomatching method for GPUrdquo in Proceedings of the 3rd Interna-tional Conference on Advanced Communications and Computa-tion pp 27ndash32 Lisbon Portugal November 2013

[48] S Lee J H Lee J Lim and I H Suh ldquoRobust stereo matchingusing adaptive random walk with restart algorithmrdquo Image andVision Computing vol 37 pp 1ndash11 2015

[49] Q Yang P Ji D Li S Yao andM Zhang ldquoFast stereo matchingusing adaptive guided filteringrdquo Image and Vision Computingvol 32 no 3 pp 202ndash211 2014

[50] H Hirschmuller P R Innocent and J Garibaldi ldquoReal-timecorrelation-based stereo vision with reduced border errorsrdquoInternational Journal of Computer Vision vol 47 no 1 pp 229ndash246 2002

[51] O Veksler ldquoFast variable window for stereo correspondenceusing integral imagesrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-556ndashI-561 IEEE Madison Wis USAJune 2003

[52] J Lu G Lafruit and F Catthoor ldquoAnisotropic local high-confidence voting for accurate stereo correspondencerdquo in ImageProcessing Algorithms and Systems VI vol 6812 of Proceedingsof SPIE pp 1ndash12 San Jose Calif USA January 2008

[53] K Zhang J Lu Q Yang G Lafruit R Lauwereins and L VanGool ldquoReal-time and accurate stereo a scalable approach withbitwise fast voting onCUDArdquo IEEETransactions onCircuits andSystems for Video Technology vol 21 no 7 pp 867ndash878 2011

[54] K-H Chen and C-L Su ldquoReducing computation complexityfor disparity matchingrdquo in Proceedings of the IEEE InternationalSymposium on Circuits and Systems (ISCAS rsquo13) pp 2916ndash2919Beijing China May 2013

[55] J Fang A L Varbanescu J Shen H Sips G Saygili and LVan Der Maaten ldquoAccelerating cost aggregation for real-timestereo matchingrdquo in Proceedings of the 18th IEEE InternationalConference on Parallel andDistributed Systems (ICPADS rsquo12) pp472ndash481 IEEE Singapore December 2012

[56] D Chen M Ardabilian and L Chen ldquoA fast trilateral filter-based adaptive support weight method for stereo matchingrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 25 no 5 pp 730ndash743 2015


[57] J Zhang J-F Nezan M Pelcat and J-G Cousin ldquoReal-timeGPU-based local stereomatchingmethodrdquo in Proceedings of the7th Conference on Design and Architectures for Signal and ImageProcessing (DASIP rsquo13) pp 209ndash214 Cagliari Italy October2013

[58] A Hosni M Bleyer and M Gelautz ldquoSecrets of adaptive sup-port weight techniques for local stereo matchingrdquo ComputerVision and Image Understanding vol 117 no 6 pp 620ndash6322013

[59] L Nalpantidis and A Gasteratos ldquoStereo vision for roboticapplications in the presence of non-ideal lighting conditionsrdquoImage and Vision Computing vol 28 no 6 pp 940ndash951 2010

[60] Z Lee J Juang and T Q Nguyen ldquoLocal disparity estimationwith three-moded cross census and advanced support weightrdquoIEEE Transactions on Multimedia vol 15 no 8 pp 1855ndash18642013

[61] C-K Liang C-C Cheng Y-C Lai L-G Chen and H HChen ldquoHardware-efficient belief propagationrdquo IEEE Transac-tions on Circuits and Systems for Video Technology vol 21 no5 pp 525ndash537 2011

[62] H-Q Wang M Wu Y-B Zhang and L Zhang ldquoEffectivestereo matching using reliable points based graph cutrdquo in Pro-ceedings of the IEEE International Conference on Visual Com-munications and Image Processing (VCIP rsquo13) pp 1ndash6 KuchingMalaysia November 2013

[63] Q Yang L Wang R Yang H Stewenius and D Nister ldquoStereomatching with color-weighted correlation hierarchical beliefpropagation and occlusion handlingrdquo IEEE Transactions onPattern Analysis andMachine Intelligence vol 31 no 3 pp 492ndash504 2009

[64] Y S Heo K M Lee and S U Lee ldquoJoint depth map and colorconsistency estimation for stereo images with different illumi-nations and camerasrdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 35 no 5 pp 1094ndash1106 2013

[65] K R Vijayanagar M Loghman and J Kim ldquoReal-time refine-ment of kinect depth maps using multi-resolution anisotropicdiffusionrdquo Mobile Networks and Applications vol 19 no 3 pp414ndash425 2014

[66] M Michael J Salmen J Stallkamp and M Schlipsing ldquoReal-time stereo vision optimizing semi-global matchingrdquo in Pro-ceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13) pp1197ndash1202 Gold Coast Australia June 2013

[67] ZMa KHe YWei J Sun and EWu ldquoConstant timeweightedmedian filtering for stereo matching and beyondrdquo in Proceed-ings of the 14th IEEE International Conference on ComputerVision (ICCV rsquo13) pp 49ndash56 IEEE Sydney Australia Decem-ber 2013

[68] A Banno and K Ikeuchi ldquoDisparity map refinement and 3Dsurface smoothing via directed anisotropic diffusionrdquo Com-puter Vision and Image Understanding vol 115 no 5 pp 611ndash619 2011

[69] S Y Park S H Lee and N I Cho ldquoSegmentation based dispar-ity estimation using color and depth informationrdquo in Proceed-ings of the International Conference on Image Processing (ICIPrsquo04) vol 5 pp 3275ndash3278 Singapore October 2004

[70] C Cigla X Zabulis and A Alatan ldquoRegion-based dense depthextraction from multi-view videordquo in Proceedings of the 14thIEEE International Conference on Image Processing (ICIP rsquo07)vol 5 pp V213ndashV216 IEEE San Antonio Tex USA September2007

[71] H Hirschmuller ldquoStereo processing by semiglobal matchingandmutual informationrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 30 no 2 pp 328ndash341 2008

[72] L Wang and R Yang ldquoGlobal stereo matching leveraged bysparse ground control pointsrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo11) pp 3033ndash3040 Providence RI USA June 2011

[73] B-Y Chen and B-C Lai ldquoA high performance parallel graphcut optimization for depth estimationrdquo inAdvances in IntelligentSystems and Applications vol 2 pp 311ndash320 Springer 2013

[74] V Kolmogorov P Monasse and P Tan ldquoKolmogorov andZabihrsquos graph cuts stereo matching algorithmrdquo Image ProcessingOn Line vol 4 pp 220ndash251 2014

[75] Y Wang F Zhong Q Peng and X Qin ldquoDepth map enhance-ment based on color and depth consistencyrdquo Visual Computervol 30 no 10 pp 1157ndash1168 2014

[76] S Ploumpis A Amanatiadis and A Gasteratos ldquoA stereomatching approach based on particle filters and scattered con-trol landmarksrdquo Image and Vision Computing vol 38 pp 13ndash232015

[77] F Tombari S Mattoccia L D Stefano and E AddimandaldquoClassification and evaluation of cost aggregation methods forstereo correspondencerdquo in Proceedings of the 26th IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo08)pp 1ndash8 Anchorage Alaska USA June 2008

[78] E T Psota J Kowalczuk J Carlson and L C Perez ldquoA localiterative refinement method for adaptive support-weight stereomatchingrdquo in Proceedings of the International Conference onImage Processing Computer Vision and Pattern Recognition pp271ndash277 Las Vegas Nev USA July 2011

[79] Q Yang ldquoA non-local cost aggregation method for stereomatchingrdquo in Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR rsquo12) pp 1402ndash1409Providence RI USA June 2012

[80] L Xu O C Au W Sun et al ldquoStereo matching by adaptiveweighting selection based cost aggregationrdquo in Proceedings ofthe IEEE International Symposium on Circuits and Systems(ISCAS rsquo13) pp 1420ndash1423 IEEE Beijing China May 2013

[81] J Koo Y-H Kim and S Lee ldquoRobust stereo matching algo-rithm for advanced driver assistance systemrdquo in Proceedingsof the IEEE International Conference on Consumer Electronics(ICCE rsquo15) pp 268ndash269 Las Vegas Nev USA January 2015

[82] T Matsuo S Fujita N Fukushima and Y Ishibashi ldquoEffi-cient edge-awareness propagation via single-map filtering foredge-preserving stereo matchingrdquo inThree-Dimensional ImageProcessing Measurement (3DIPM) and Applications vol 9393of Proceedings of SPIE International Society for Optics andPhotonics San Francisco Calif USA March 2015

[83] L Nalpantidis andA Gasteratos ldquoBiologically and psychophys-ically inspired adaptive support weights algorithm for stereocorrespondencerdquoRobotics and Autonomous Systems vol 58 no5 pp 457ndash464 2010

[84] M Werner B Stabernack and C Riechert ldquoHardware imple-mentation of a full HD real-time disparity estimation algo-rithmrdquo IEEE Transactions on Consumer Electronics vol 60 no1 pp 66ndash73 2014

[85] D F Bacon R Rabah and S Shukla ldquoFPGA programming forthe massesrdquo Communications of the ACM vol 56 no 4 pp 56ndash63 2013

[86] N Wilt The Cuda Handbook A Comprehensive Guide to GpuProgramming Pearson Education 2013


[87] R Kalarot and J Morris ldquoComparison of FPGA and GPUimplementations of real-time stereo visionrdquo in Proceedings ofthe IEEE Computer Society Conference on Computer Vision andPattern Recognition Workshops pp 9ndash15 IEEE San FranciscoCalif USA June 2010

[88] K Pauwels M Tomasi J Dıaz Alonso E Ros and M M VanHulle ldquoA Comparison of FPGA and GPU for real-time phase-based optical flow stereo and local image featuresrdquo IEEE Trans-actions on Computers vol 61 no 7 pp 999ndash1012 2012

[89] T Xu P Cockshott and S Oehler ldquoAcceleration of stereo-matching on multi-core CPU and GPUrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications pp 108ndash115 IEEE Paris France August2014

[90] LM Russo E C Pedrino E Kato and V O Roda ldquoImage con-volution processing a GPU versus FPGA comparisonrdquo in Pro-ceedings of the 8th Southern Programmable Logic Conference(SPL rsquo12) pp 1ndash6 Bento Goncalves Brazil March 2012

[91] J Fowers G Brown J Wernsing and G Stitt ldquoA performanceand energy comparison of convolution on GPUs FPGAs andmulticore processorsrdquo ACM Transactions on Architecture andCode Optimization vol 9 no 4 article 25 2013

[92] M Jin andTMaruyama ldquoFast and accurate stereo vision systemon FPGArdquoACMTransactions on Reconfigurable Technology andSystems vol 7 no 1 article 3 2014

[93] G Lentaris I Stamoulias D Diamantopoulos et al ldquoSPAR-TANSEXTANTCOMPASS advancing space rover vision viareconfigurable platformsrdquo in Applied Reconfigurable Comput-ing pp 475ndash486 Springer 2015

[94] W Wang J Yan N Xu Y Wang and F-H Hsu ldquoReal-timehigh-quality stereo vision system in FPGArdquo in Proceedingsof the 12th International Conference on Field-ProgrammableTechnology (FPT rsquo13) pp 358ndash361 Kyoto JapanDecember 2013

[95] VMota G FalcaoMAntunes J Barreto andUNunes ldquoUsingthe GPU for fast symmetry-based dense stereo matching inhigh resolution imagesrdquo in Proceedings of the IEEE InternationalConference on Acoustics Speech and Signal Processing (ICASSPrsquo14) pp 7520ndash7524 Florence Italy May 2014

[96] Y-K Choi and I K Park ldquoEfficient GPU-based graph cutsfor stereo matchingrdquo in Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition Workshops (CVPRWrsquo13) pp 642ndash648 IEEE Portland Ore USA June 2013

[97] S-J Yao L-H Wang C-L Lin and M Zhang ldquoReal-timestereo to multi-view conversion system based on adaptivemeshingrdquo Journal of Real-Time Image Processing pp 1ndash19 2015

[98] M G Samarawickrama Performance Evaluation of VisionAlgorithms on FPGA Universal-Publishers 2010

[99] S Mattoccia ldquoStereo vision algorithms for FPGAsrdquo in Proceed-ings of the IEEE Conference on Computer Vision and PatternRecognition Workshops (CVPRW rsquo13) pp 636ndash641 PortlandOre USA June 2013

[100] C Colodro-Conde F J Toledo-Moreo R Toledo-Moreo J JMartınez-Alvarez J Garrigos Guerrero and J M Ferrandez-Vicente ldquoEvaluation of stereo correspondence algorithms andtheir implementation on FPGArdquo Journal of Systems Architec-ture vol 60 no 1 pp 22ndash31 2014

[101] F Tombari F Gori and L Di Stefano ldquoEvaluation of stereoalgorithms for 3D object recognitionrdquo in Proceedings of the IEEEInternational Conference on Computer VisionWorkshops (ICCVrsquo11) pp 990ndash997 IEEE Barcelona Spain November 2011

[102] K-J Yoon and I S Kweon ldquoAdaptive support-weight approachfor correspondence searchrdquo IEEE Transactions on Pattern Anal-ysis and Machine Intelligence vol 28 no 4 pp 650ndash656 2006

[103] Y Zhao and G Taubin ldquoReal-time stereo on GPGPU usingprogressive multi-resolution adaptive windowsrdquo Image andVision Computing vol 29 no 6 pp 420ndash432 2011

[104] J Kowalczuk E T Psota and L C Perez ldquoReal-time stereomatching on CUDA using an iterative refinement method foradaptive support-weight correspondencesrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 23 no 1 pp94ndash104 2013

[105] Y Lin N Lu X Lou F Zou Y Yao and Z Du ldquoMatchingcost filtering for dense stereo correspondencerdquo MathematicalProblems in Engineering vol 2013 Article ID 654139 11 pages2013

[106] B Tippetts D J Lee K Lillywhite and J K ArchibaldldquoHardware-efficient design of real-time profile shape matchingstereo vision algorithm on FPGArdquo International Journal ofReconfigurable Computing vol 2014 Article ID 945926 12pages 2014

[107] J-S Kim M Hwangbo and T Kanade ldquoParallel algorithms toa parallel hardware designing vision algorithms for a GPUrdquoin Proceedings of the IEEE 12th International Conference onComputer Vision Workshops (ICCV rsquo09) pp 862ndash869 KyotoJapan October 2009

[108] E S L Gastal and M M Oliveira ldquoDomain transform foredge-aware image and video processingrdquo ACM Transactions onGraphics vol 30 no 4 article 69 2011

[109] J Fowers G Brown P Cooke and G Stitt ldquoA perfor-mance and energy comparison of FPGAs GPUs and multi-cores for sliding-window applicationsrdquo in Proceedings of theACMSIGDA International Symposium on Field ProgrammableGate Arrays (FPGA rsquo12) pp 47ndash56 ACMMonterey Calif USAFebruary 2012

[110] F Ekstrand C Ahlberg M Ekstrom and G SpampinatoldquoTowards an embedded real-time high resolution vision sys-temrdquo in Advances in Visual Computing pp 541ndash550 SpringerBerlin Germany 2014

[111] M Perez-Patricio A Aguilar-Gonzalez J L Camas-Anzuetoand M Arias-Estrada ldquoA fuzzy logic approach for stereomatching suited for real-time processingrdquo International Journalof Computer Applications vol 113 no 2 pp 1ndash8 2015

[112] F Stein ldquoThe challenge of putting vision algorithms into acarrdquo in Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition Workshops (CVPRWrsquo12) pp 89ndash94 IEEE Providence RI USA June 2012

[113] Q Long Q Xie S Mita H Tehrani K Ishimaru and CGuo ldquoReal-time dense disparity estimation based onmulti-pathviterbi for intelligent vehicle applicationsrdquo in Proceedings of theBritish Machine Vision Conference pp 1ndash13 BMVA Press 2014

[114] A Geiger P Lenz C Stiller and R Urtasun ldquoTheKITTI VisionBenchmark Suiterdquo 2015

[115] M Chessa F Solari and S P Sabatini ldquoDatasets for disparityand optic flow evaluationrdquo 2015 httpwwwpspcunigeit

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



Quantity of original papers from ScienceDirect

22 23 39

32

33

43

50

58

41

47

YearTrendline

0

10

20

30

40

50

60

70

Qua

ntity

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

Journalbook

(a)

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

YearTrendline

Quantity of original papers from IEEE Xplore

92

144

144

165

198

207

186

224

168

171

0

50

100

150

200

250

Qua

ntity

Conferencejournal

(b)

Figure 1 The yearly distributions of the number of papers published from 2005 to 2014 in the area of stereo vision from the database of (a)ScienceDirect and (b) IEEE Xplore

vision sensors (ie passive sensors) are more reliable androbust they are able to produce high-resolution disparitymaps and are suitable for both indoor and outdoor environ-ments [7]

In stereo vision disparity map processing the num-ber of calculations required increases with an increasingnumber of pixels per image This phenomenon causes thematching problem to be computationally complex [8] Theimprovements to and reduction in computational complexitythat have been achieved with recent advances in hardwaretechnology have been beneficial for the advancement ofresearch in the stereo vision field Thus the main motivationfor hardware-based implementation is to achieve real timeprocessing [9] In real time stereo vision applications suchas autonomous driving 3D gaming and autonomous roboticnavigation fast but accurate depth estimations are required[10] Additional processing hardware is therefore necessaryto improve the processing speed

An updated survey on stereo vision disparity map algo-rithms would be valuable to those who are interested in thisresearch area Figures 1(a) and 1(b) illustrate the quantityof original contributions published in this area over thepast ten years (ie 2005ndash2014) from the databases of Sci-enceDirect and IEEE Xplore The keywords used were stereovisionstereo vision algorithm and the components that weresearchedwere the title abstract and keywordsindex terms ofthe papers in the databases All of these papers may representcontributions to fundamental algorithm development analy-sis or application of stereo vision algorithms In both figuresthe trendlines are increasing indicating that the field ofstereo vision remains active in research and development andhas become an interesting and challenging area of researchThis paper provides a brief introduction to the state-of-the-art developments accomplished in the context of suchalgorithms This work reviews the latest published stereovision algorithms and categorizes them into different stagesof processing which are based on the taxonomy proposedby Scharstein and Szeliski [11] This paper also discusses

two types of implementation platforms for these algorithms(ie software-based and hardware-based) In software-basedplatforms the techniques are implemented only on a standardCPU without any other additional processing hardware Incontrast to hardware-based platforms the algorithms areexecuted on a CPU with a GPU or FPGA as a standalonesystem

The remainder of the paper is organized as followsPrevious review papers related to stereo vision disparity mapalgorithms are discussed in Section 2Then the taxonomy forthe stages of processing performed in stereo vision disparitymap algorithms is presented in Section 3 It consists of foursubsections (ie matching cost computation cost aggre-gation disparity selection and optimization and disparityrefinement) Section 4 presents a review of algorithms imple-mented through software-based platforms and Section 5discusses real time stereo vision disparity map algorithmsbased on additional hardware (ie FPGAs and GPUs) Amethod ofmeasuring the accuracy of stereo vision algorithmsis explained in Section 6 and the conclusion is presented inSection 7

2 Previous Reviews of Stereo VisionDisparity Map Algorithms

Numerous methods of implementation for stereo visiondisparity mapping have been established in the past fewyears This can be observed from the review papers listedin Table 1 The contents of these review papers are alsosummarized in this table Among these review papers themain focus was to summarize and compare the accuracylevel and execution time of each cited algorithm Howevernone of these reviews provided a detailed discussion of thestages of implementation based on the taxonomy proposedby Scharstein and Szeliski [11] as does the survey presentedin this paper Furthermore this paper also reviews thelatest algorithms implemented using two different types ofplatforms (ie software-based and hardware-based)



Year Author Focus








Inputimages

Step 1

Matchingcost

computation

Step 2

Costaggregation

Step 3

Disparityselection

Step 4

Disparityrefinement

Disparitymap








P

Il Ir

pl pre e

OlOr







AD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (1)



SD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (2)





SAD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (3)





SSD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (4)



NCC (119909 119910 119889)

=


radicsum(119909119910)isin119908 1198682

119897(119909 119910) sdot sum

(119909119910)isin1199081198682119903(119909 minus 119889 119910)

(5)



Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash


ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











Year Author Focus








Inputimages

Step 1

Matchingcost

computation

Step 2

Costaggregation

Step 3

Disparityselection

Step 4

Disparityrefinement

Disparitymap








P

Il Ir

pl pre e

OlOr







AD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (1)



SD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (2)





SAD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (3)





SSD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (4)



NCC (119909 119910 119889)

=


radicsum(119909119910)isin119908 1198682

119897(119909 119910) sdot sum

(119909119910)isin1199081198682119903(119909 minus 119889 119910)

(5)



Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash


ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










P

Il Ir

pl pre e

OlOr







AD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (1)



SD (119909 119910 119889) = 1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (2)





SAD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (3)





SSD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (4)



NCC (119909 119910 119889)

=


radicsum(119909119910)isin119908 1198682

119897(119909 119910) sdot sum

(119909119910)isin1199081198682119903(119909 minus 119889 119910)

(5)



Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash


ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












SAD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816 (3)





SSD (119909 119910 119889) = sum(119909119910)isin119908

1003816100381610038161003816119868119897 (119909 119910) minus 119868119903 (119909 minus 119889 119910)1003816100381610038161003816

2 (4)



NCC (119909 119910 119889)

=


radicsum(119909119910)isin119908 1198682

119897(119909 119910) sdot sum

(119909119910)isin1199081198682119903(119909 minus 119889 119910)

(5)



Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash


ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Table2Summaryof

stereovisio

ndisparity

map

algorithm

simplem

entedon

different

compu

tatio

nalplatfo

rms

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2006

Wangetal[22]

Local

AD

MW

WTA

Medianfilter

Multip

lecostaggregation

CPUIntelP

4andGPU

ATI

800

2006

Yoon

andKw

eon[102]Lo

cal

TAD

ASW

WTA

Left-rig

htconsistency

check

Adaptiv

esup

portweight

mdash

2007

Ciglae

tal[70]

Local

Edge

based

Arbitraryshaped

basedAW

WTA

Ang

lesw

eep

Multiv

iewregion

-based

mdash

2008

Hirs

chmuller[71]

Global

Semiglobal

matchingpixel

based

mdashDP

Left-rig

htconsistency

check

+medianfilterin

gSemiglobalm

atching

CPUOpteron

22G

Hz

2009

Yang

etal[63]

Global

Color

weighted

correlation

mdashBP

Left-rig

htconsistency

check

Hierarchicalloo

pybelief

prop

agation

mdash

2009

Salm

enetal[16]

Global

SAD

mdashDP

Left-rig

htconsistency

check

Optim

ized

dynamic

programming

CPUwith

18GHz

2010

Mattocciaetal[13]

Local

TAD

Jointb

ilateral

filterin

gbased

ASW

WTA

Left-rig

htconsistency

check

Jointb

ilateralstereo

CPUIntelC

oreD

uo214GHz

2010

Nalpantidisand

Gasteratos[5983]

Local

AD

ASW

WTA

mdashLC

DM

mdash

2010

Kalarotand

Morris

[87]

Global

Pixelbased

mdashDP

mdashSymmetric

dynamic

programming

CPUIntelQ

uadCore

23G

Hza

ndGPU

GeForce

GTX

280FP

GAAltera

Stratix

III

2011

Leea

ndSharma[

33]

Local

SAD+RT

Multiresolution

basedMW

WTA

mdashRa

nktransfo

rm+

multiresolution

Person

alCom

putera

ndGPU

(detailn

a)

2011

Meietal[45]

Local

AD+CT

Crossb

ased

AWWTA

Cross-checking

ADcensus

+dynamiccross

CPUIntelC

ore2

Duo

220

GHza

ndGPU

GeForce

GTX

480

2011

Min

etal[23]

Local

TAD

Jointh

istogram-

MW

based

aggregation

WTA

Boxfilterin

gHistogram-based

aggregation

CPU(detailn

a)

2011

Pereza

ndSanchez

[17]

Global

Pixelbased

mdashBP

Left-rig

htconsistency

check

Mem

ory-effi

cientb

elief

prop

agation

mdash

2011

Sharmae

tal[28]

Local

Featureb

ased

mdashWTA

mdashSIFT

Autono

mou

svehicle

navigatio

n

2011

Psotae

tal[78]

Local

SAD

ASW

WTA

Medianfilter

Iterativ

erefinement

adaptiv

esup

portweight

mdash


ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










ble2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2011

Tipp

ettsetal[32]

Local

Gaussianfilter+

profi

leshape

MW

WTA

Medianfilter

Profi

leshapem

atching

CPUAMDPh

enom

II28G

Hz

2011

Bann

oandIkeuchi

[68]

Global

Pixelbased

mdashBP

Dire

cted

aniso

tropic

diffu

sion

3Ddirected

aniso

tropic

diffu

sion

mdash

2011

Zhao

andTaub

in[103]

Local

SAD

Multiresolution

basedAW

WTA

Cross-checking


ewindo

ws

CPU(detailn

a)a

ndGPU

GeForce

GTX

280

2011

Nalpantidisetal[20]

Local

AD

Gaussianweighted

+cellu

lara

utom

ata

WTA

mdashEffi

cienth

ierarchical

Stratix

IIGXFP

GA

2012

Arranze

tal[14

]Global

Pixelbased

mdashMRF

based

optim

ization


minim

ization

CPUInteli734G

Hzw

ith7G

BRA

M

2012

Xiangetal[9]

Global

Color

weighted

pixelbased

mdashHierarchical

belief

prop

agation

Uniqu

eness

constraint

+sim

ilarity

constraint

Hierarchicalbelief

prop

agation

CPUIntel27GHza

ndGPU

Geforce

GTX

260

2012

Yang

[79]

Local

Edge

based

Minim

umspanning

tree

WTA

Guidedfilter

Minim

umspanning

tree

based

CPUInteli718

GHzw

ith4G

BRA

M2013

Chen

andLai[73]

Global

Pixelbased

mdashGC

mdashParallelG

CCP

UInteli7(detailn

a)

2013

Chen

andSu

[54]

Local

Areab

ased

Shapea

daptivelow

complexity

based

AWWTA

Cross-checking

Shapea

daptivelow

complexity

CPUInteli526G

Hz

2013

Wangetal[18]

Global

Pixelbased

mdashGC


disparity

structure

(HBD

S)CP

UIntelC

ore2

Duo

233

GHzw

ith2G

BRA

M

2014

Chengetal[40

]Lo

cal+

glob

alZN

CCMatch-edgeb

ased

MW

WTA

Globaledge

constraint

Globaledgec

onstr

aint

na

2013

ChoiandPark

[96]

Global

SAD

mdashGC

Trun

catedlin

ear

functio

nEffi

cientG

PU-based

GC

CPU(detailn

a)a

ndGPU

GeForce

GTX

580

2013

Wangetal[62]

Global

TAD

mdashGC

Medianfilterin

g+

interpolationand

depth

discon

tinuity

adjustment

Reliablep

ointsb

ased

GC

CPUIntelC

ore2

Duo

20G

Hzw

ith2G

BRA

M

2013

Michaeletal[66

]Lo

cal+

glob

alAD

mdashWTA

Left-rig

htconsistency

check

Realtim

esem

iglobal

CPUIntelX

eonandGPU

GeForce

GTX

480

2013

Ciglaa

ndAlatan

[46]

Local

SAD+CT

Successiv

eweighted

summationbased

ASW

WTA

Perm

eabilitybased

filterin

gSeparables

uccessive

weightedsummation

CPUInteli73G

Hzw

ith6G

BRA

Mand

GPU

GeForce

GTX

480

2013

Gup

taandCh

o[34]

Local

SAD

MW

WTA

Left-rig

htconsistency

check

Dou

blew

indo

wbased

CPU(detailn

a)

2013

Hosni

etal[58]

Local

TAD(color

+gradient)

ASW

WTA

Weightedbilateral

medianfilterin

gASW

+patchmatch

stereo

CPUIntelC

ore2

Quad

24G

HZandGPU

GeForce

GTX

480


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2013

Zhangetal[57]

Local

TAD(color

+gradient)

ASW

WTA

mdashRe

altim

eGPU

CPUInteli7andGPU

GeForce

GT5

40M

2013

Kowalczuk

etal[104]

Local

SAD

Iterativ

eASW

+tempo

ralA

SWWTA

Medianfilterin

gIte

rativ

erefinementA

SWCP

UAMDPh

enom

IIX6

30G

Hza

ndGPU

GeForce

GTX

580

2013

Leee

tal[60]

Local

Three-mod

ecross

CTASW

WTA

Left-rig

htconsistency

check

ASW

+three-mod

ecross

census

mdash

2013

Linetal[105]

Local

TAD(color

+gradient)

Hierarchical

cluste

rbased

ASW

WTA

Left-rig

htconsistency

check

Hierarchicalcluste

ring

algorithm

CPUIntelC

ore2

Duo

299

GHza

ndGPU

GeForce

9500GT

2013

Zhangetal[47]

Local

SAD+ALD

Crossb

ased

AWWTA

Left-rig

htconsistency

check

SAD+arm-le

ngth

differences

(ALD

)

CPUIntelC

ore2

Duo

300

GHza

ndGPU

GeForce

GTX

570

2013

Pham

andJeon

[24]

Local

TAD

MW

WTA

Weightedmedian

filterin

gDom

aintransfo

rmation

CPUAMDAthlon

64X2

DualC

ore2

GHza

ndGPU

GeForce

GTX

460

2014

Vijayanagare

tal[65]

Local

Segm

entb

ased

MW

WTA

Multiresolutio

naniso

tropic

diffu

sion


tropic

diffu

sion

CPUrunn

ingat32G

Hz

with

8GBRA

MandGPU

AMDFireProV4900

2013

Xuetal[1580]

Local

TAD

ASW

WTA

mdashAd

aptiv

eweightselectio

nmdash

2014

Wangetal[75]

Global

Pixelbased

mdashMRF

based

optim

ization

Jointb

ilateral

filterin

gMRF

basedop

timization

CPUrunn

ingat3G

Hz

with

3GBRA

M

2013

Mattoccia[99]

Local

Segm

entb

ased

ASW

WTA

Guidedfilter

Multip

lealgorithm

s(FW

ASW

)FP

GAXilin

xSpartan6

2013

Zhangetal[6]

Global

BT+AD

mdashBP

Color

imaged

epth

mattin

g+2D

polyno

mial

regressio

n

MultiscaleBP

CPUInteli7with

4GBRA

MandGPU

GeForce

GTX

295

2013

Fowerse

tal[91]

Local

SAD

MW

WTA

mdashSA

D

CPUIntelQ

uadCore

300

GHza

ndGPU

GeForce

GTX

295FP

GA

Altera

Stratix

IIIE

260

2013

Wangetal[94]

Local

AD+CT

AWWTA

mdashRe

altim

eHQ-FPG

AFP

GAAltera

Stratix

IV

2013

Mae

tal[4467]

Local

TAD

ASW

WTA

Weightedmedian

filterin

g+left-rig

htconsistency

check

Con

stant

timew

eighted

medianfilterin

gCP

UIntelX

eon283

GHz

andGPU

GeForce

GTX

580

2014

Miro

netal[26]

Local

CTCr

ossb

ased

AWWTA

Anisotro

piclocal

polyno

mial

approxim

ation

Cross-comparis

oncensus

mdash


Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Table2Con

tinued

Year

Author

Metho

dTaxono

mysta

geAlgorith

mCom

putatio

nalplatfo

rmStep1

Step2

Step3

Step4

2014

Yang

etal[49]

Local

TAD

AWWTA

Adaptiv

eguided

filter

Fastadaptiv

eguidedfilter

CPUIntelC

oreD

uo316GHzw

ith2G

BRA

MandGPU

NvidiaT

esla

C2050

2014

Colod

ro-C

onde

etal

[100]

Local

SAD+CT

MW

WTA

Medianfilterin

gSA

D+CT

Xilin

xFP

GA

2014

JinandMaruyam

a[92]

Local

CEN+AD

AWWTA

mdashCA

+FL

CXilin

xXC

6VLX

240T

2004

Park

etal[69]

Local+

glob

alEd

gebased

FWMRF

based

Boxfilterin

gSegm

entatio

nbased

mdash

2014

Motae

tal[95]

Global

Symmetry

mdashMRF

based

mdashSymStereo

CPUInteli73G

Hza

ndGeforce

GTX

680

2014

Tipp

ettsetal[106]

Local

Edge

MW

WTA

Verticalfilterin

gProfi

leshapem

atching

Xilin

xVirtex

4FX

60

2014

Xuetal[89]

Local

Pixelbased

FWWTA

mdashPy

ramidal

AMDOpteron

processor

6366HE

IntelX

eon

processorE

5-2620N

vidia

GeForce

GTX

770

2014

Wernere

tal[84]

Local

NCC

mdashWTA

Crossb

ilateralfilterRe

altim

efullH

DFP

GA

2015

Ekstr

andetal[31]

Local

Segm

entatio

nMW

WTA

Medianfilter

Realtim

esegmentatio

nXilin

xFP

GA-

Spartan

3A-1800A

2015

Matsuoetal[82]

Local

AD+edge

FWWTA

Weightedjoint

bilateralfi

lter

In-Loo

pFeedback

Matching

mdash

2015

Yaoetal[97]

Global

Feature-based

mdashMRF

mdashRe

altim

eadaptivem

eshing

CPUXe

onandNvidia

Tesla

K2

2015

Plou

mpise

tal[76]

Global

Feature+

particle

filter

mdashMRF

based

Histogram

+segm

entatio

nParticlefilter+

scattered

controllandm

arks

CPUIntelC

ore2

Duo

2015

Chen

etal[56]

Local

Medianfilter+

AD

ASW

WTA

mdashTrilateral

CPUInteli73G

Hzw

ith8G

RAM




RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












RT (119909 119910 119889)

= sum

(119909119910)isin119908


(6)


Rank (119909 119910) = sum(119894119895)(119909119910)

119871 (119894 119895) (7)


119871 (119894 119895) =

0 119868 (119894 119895) lt 119868 (119909 119910)

1 otherwise(8)





CT (119909 119910 119889) = sum(119909119910)isin119908






(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b)

(d)

(c)

Target pixel

Subw

indow







1198621015840(119901 119889) = sum

119902isin119908119901

119882(119901 119902) 119862 (119902 119889) (11)





119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












119889119901 = arg min119889isin119863

1198621015840(119901 119889) (12)




119864 (119889) = 119864data (119889) + 120573119864smooth (119889) (13)






CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










CPU


OpenCL etc)

(a)

CPU

Libraries

GPU


(b)

CPU

FPGA

VHDLVerilog etc

(c)















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation

















2008 Gac et al [41]





























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation
























Tsukuba

Venus

Teddy

Cones







PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










PM-F

ores

tSe

gAgg

rLC

UA

DC

ensu

sPM

FSu

bPix

Sear

chSO

SSu

bPix

Dou

bleB

PRD

PPa

tchM

atch

CCRA

DA

RLA

MC-

DSM

Adap

tiveG

FH

istA

ggrS

lant

Mul

tiAgg

Tree

Filte

rRe

altim

eED

PLM

3CCr

ossL

MF

Hist

oAgg

r2

impr

NLC

AD

TAgg

r-P

Hyb

ridTr

eeSe

gmen

tTre

eRe

lativ

eGra

d

HEB

FBS

M

AlgorithmLocal

AllNonocc Disc

Cros

sTre

es+

SP

Segm

+vi

sib

TF_A

SW

000200400600800

1000

Bad

pixe

l (

)




Global

AllNonocc Disc

LLR

CSM

TwoS

tep

IGSM

ARA

P

PMBP

TSG

O

GC+

LSL

SCoB

eP

MVS

egBP

GCP

MC

War

pMat

SegT

reeD

P

Dou

bleB

P

FastN

LGC

CVW

-RM

Plan

eFitB

P

Obj

ectS

tere

o

Art

istic

GC

JSO

SP+

GCP

Glo

balG

CP

Adap

tingB

P

Mul

tiRes

GC

Surfa

ceSt

ereo

Coo

pReg

ion

Real

timeH

D

Inte

riorP

tLP

Out

lierC

onf

Impr

oveS

ubPi

x

BP+

Dire

cted

Diff

Algorithm

000200400600800

100012001400

Bad

pixe

l (

)


1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Algorithm


000100200300400500600700800

Bad

pixe

l (

)


7 Conclusion


Nomenclature





support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












support window



Acknowledgments


References

























































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation








































































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









review article literature survey on stereo vision

Documents