Differential angular imaging for material recognition

Jia Xue; Hang Zhang; Kristin Dana; Ko Nishino

doi:https://doi.org/10.1109/CVPR.2017.734

Differential angular imaging for material recognition

Jia Xue, Hang Zhang, Kristin Dana, Ko Nishino

School of Engineering, Electrical & Computer Engineering

Rutgers, The State University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

34 Scopus citations

Abstract

Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation “in the wild.” Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose a middle-ground approach that takes advantage of both rich radiometric cues and flexible image capture. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS), geared towards real use for autonomous agents. This publicly available database¹ consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.

Original language	English (US)
Title of host publication	Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	6940-6949
Number of pages	10
ISBN (Electronic)	9781538604571
DOIs	https://doi.org/10.1109/CVPR.2017.734
State	Published - Nov 6 2017
Event	30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States Duration: Jul 21 2017 → Jul 26 2017

Publication series

Name	Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
Volume	2017-January

Other

Other	30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
Country/Territory	United States
City	Honolulu
Period	7/21/17 → 7/26/17

ASJC Scopus subject areas

Signal Processing
Computer Vision and Pattern Recognition

Access to Document

https://doi.org/10.1109/CVPR.2017.734

Cite this

Xue, J., Zhang, H., Dana, K., & Nishino, K. (2017). Differential angular imaging for material recognition. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (pp. 6940-6949). (Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017; Vol. 2017-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CVPR.2017.734

@inproceedings{f11f63bc88de4de896f4d71719fd1f70,

title = "Differential angular imaging for material recognition",

abstract = "Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation “in the wild.” Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose a middle-ground approach that takes advantage of both rich radiometric cues and flexible image capture. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS), geared towards real use for autonomous agents. This publicly available database1 consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.",

author = "Jia Xue and Hang Zhang and Kristin Dana and Ko Nishino",

note = "Funding Information: This work was supported by National Science Foundation award IIS-1421134. A GPU used for this research was donated by the NVIDIA Corporation. Thanks to Di Zhu, Hansi Liu, Lingyi Xu, and Yueyang Chen for help with data collection. Publisher Copyright: {\textcopyright} 2017 IEEE.; 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 ; Conference date: 21-07-2017 Through 26-07-2017",

year = "2017",

month = nov,

day = "6",

doi = "https://doi.org/10.1109/CVPR.2017.734",

language = "English (US)",

series = "Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "6940--6949",

booktitle = "Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017",

address = "United States",

}

Xue, J, Zhang, H, Dana, K & Nishino, K 2017, Differential angular imaging for material recognition. in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, vol. 2017-January, Institute of Electrical and Electronics Engineers Inc., pp. 6940-6949, 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, United States, 7/21/17. https://doi.org/10.1109/CVPR.2017.734

Differential angular imaging for material recognition. / Xue, Jia; Zhang, Hang; Dana, Kristin et al.
Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 6940-6949 (Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017; Vol. 2017-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Differential angular imaging for material recognition

AU - Xue, Jia

AU - Zhang, Hang

AU - Dana, Kristin

AU - Nishino, Ko

N1 - Funding Information: This work was supported by National Science Foundation award IIS-1421134. A GPU used for this research was donated by the NVIDIA Corporation. Thanks to Di Zhu, Hansi Liu, Lingyi Xu, and Yueyang Chen for help with data collection. Publisher Copyright: © 2017 IEEE.

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation “in the wild.” Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose a middle-ground approach that takes advantage of both rich radiometric cues and flexible image capture. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS), geared towards real use for autonomous agents. This publicly available database1 consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.

AB - Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation “in the wild.” Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose a middle-ground approach that takes advantage of both rich radiometric cues and flexible image capture. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS), geared towards real use for autonomous agents. This publicly available database1 consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.

UR - http://www.scopus.com/inward/record.url?scp=85044366639&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044366639&partnerID=8YFLogxK

U2 - https://doi.org/10.1109/CVPR.2017.734

DO - https://doi.org/10.1109/CVPR.2017.734

M3 - Conference contribution

T3 - Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

SP - 6940

EP - 6949

BT - Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Y2 - 21 July 2017 through 26 July 2017

ER -

Xue J, Zhang H, Dana K, Nishino K. Differential angular imaging for material recognition. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 6940-6949. (Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017). doi: https://doi.org/10.1109/CVPR.2017.734

Differential angular imaging for material recognition

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this