Published Papers - AMANO Toshiyuki

Separating the Direct and Global Components of a Single Image.

Art Subpa-Asa, Ying Fu, Yinqiang Zheng, Toshiyuki Amano, Imari Sato

JIP   26   755 - 767   2018  [Refereed]

DOI

Light Projection-Induced Illusion for Controlling Object Color.

Ryo Akiyama, Goshiro Yamamoto, Toshiyuki Amano, Takafumi Taketomi, Alexander Plopski, Christian Sandor, Hirokazu Kato

2018 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2018, Tuebingen/Reutlingen, Germany, 18-22 March 2018 ( IEEE Computer Society )    499 - 500   2018  [Refereed]

DOI

Viewpoint-Dependent Appearance-Manipulation with Multiple Projector-Camera Systems.

Toshiyuki Amano, Shun Ushida, Yusuke Miyabayashi

International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments, ICAT-EGVE 2017, Adelaide, Australia, November 22 - 24, 2017. ( Eurographics Association )    101 - 107   2017.11  [Refereed]

DOI

知覚量に基づく光投影による色制御実現に向けた色知覚モデルの検討

秋山諒, 山本豪志朗, 天野敏之, 武富貴史, Alexander Plopski, Christian Sandor, 加藤博一

日本バーチャルリアリティ学会複合現実感研究会     MR2017-16   2017.10

Adaptive Appearance Manipulation for the Installation Art

Toshiyuki Amano

JSAP-OSA Joint Symposia 2017     p. 6a-A409-5 - p. 6a-A409-5   2017.09  [Invited]

3-2 Manipulation of Material Perception by the Projection Mapping and Novel Representation of Gold and Silver Leaf art Beauty

Amano Toshiyuki, Rakusho Hiroto

The Journal of The Institute of Image Information and Television Engineers ( 一般社団法人 映像情報メディア学会 )  71 ( 1 ) 30 - 35   2017.01  [Invited]

Separation of Reflective and Fluorescent Components using Projector-Camera System

島名功, 天野敏之

日本バーチャルリアリティ学会論文誌(Web) ( THE VIRTUAL REALITY SOCIETY OF JAPAN )  22 ( 3 ) 421‐429(J‐STAGE) - 429   2017  [Refereed]

　View Summary

<p>Fluorescence can be a beneficial property not only for the scene analysis, but also for a spatial augmented reality (SAR). In the field of SAR, the projector camera system has been well studied and its radiometric model can be described by a color mixing matrix. For the measurement of the color mixing matrix, special equipments other than a projector and a camera are not required. The fluorescent components are not separated by the color mixing matrix, but it already has a capability of handling light emission of fluorescent. In this paper, we propose the RKS model that expresses fluorescent components as well as reflectance components from the approximation of the Donaldson matrix. In addition, we propose a decomposition method of the color mixing matrix. Experimental results show the decomposition of color mixing matrices obtained from color construction papers and non-fluorescent Kent papers with several fluorescent paintings. Then, we demonstrate separation of reflectance and fluorescent components of a scene. We also demonstrate effectiveness of radiometric compensation that considers fluorescent components.</p>

DOI

Monocular Focus Estimation Method for a Freely-Orienting Eye using Purkinje-Sanson Images

Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, Toshiyuki Amano, Maki Sugimoto

2017 IEEE VIRTUAL REALITY (VR) ( IEEE COMPUTER SOC )    213 - 214   2017  [Refereed]

　View Summary

We present a method for focal distance estimation of a freelyorienting eye using Purkinje-Sanson (PS) images, which are reflections of light on the inner structures of the eye. Using an infrared camera with a rigidly-fixed LED, our method creates an estimation model based on 3D gaze and the distance between reflections in the PS images that occur on the corneal surface and anterior surface of the eye lens. The distance between these two reflections changes with focus, so we associate that information to the focal distance on a user. Unlike conventional methods that mainly relies on 2D pupil size which is sensitive to scene lighting and the fourth PS image, our method detects the third PS image which is more representative of accommodation. Our feasibility study on a single user with a focal range from 15-45 cm shows that our method achieves mean and median absolute errors of 3.15 and 1.93 cm for a 10-degree viewing angle. The study shows that our method is also tolerant against environment lighting changes.

DOI

3D Single/Multiple Ground Planes Detection with Camera Angle Estimation

LANG Yankun, WU Haiyuan, AMANO Toshiyuki, CHEN Qian

IIEEJ transactions on image electronics and visual computing ( The Institute of Image Electronics Engineers of Japan )  5 ( 1 ) 42 - 51   2017  [Refereed]

Direct and global component separation from a single image using basis representation

Art Subpa-Asa, Ying Fu, Yinqiang Zheng, Toshiyuki Amano, Imari Sato

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) ( Springer Verlag )  10113   99 - 114   2017  [Refereed]

　View Summary

Previous research showed that the separation of direct and global components could be done with a single image by assuming neighboring scene points have similar direct and global components, but it normally leads to loss of spatial resolution of the results. To tackle such problem, we present a novel approach for separating direct and global components of a scene in full spatial resolution from a single captured image, which employs linear basis representation to approximate direct and global components. Due to the basis dependency of these two components, high frequency light pattern is utilized to modulate the frequency of direct components, which can effectively improve stability of linear model between direct and global components. The effectiveness of our approach is demonstrated on both simulated and real images captured by a standard off-the-shelf camera and a projector mounted in a coaxial system. Our results show better visual quality and less error compared with those obtained by the conventional single-shot approach on both still and moving objects.

DOI

Direct and global component separation from a single image using basis representation

Art Subpa-Asa, Ying Fu, Yinqiang Zheng, Toshiyuki Amano, Imari Sato

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) ( Springer Verlag )  10113   99 - 114   2017  [Refereed]

　View Summary

Previous research showed that the separation of direct and global components could be done with a single image by assuming neighboring scene points have similar direct and global components, but it normally leads to loss of spatial resolution of the results. To tackle such problem, we present a novel approach for separating direct and global components of a scene in full spatial resolution from a single captured image, which employs linear basis representation to approximate direct and global components. Due to the basis dependency of these two components, high frequency light pattern is utilized to modulate the frequency of direct components, which can effectively improve stability of linear model between direct and global components. The effectiveness of our approach is demonstrated on both simulated and real images captured by a standard off-the-shelf camera and a projector mounted in a coaxial system. Our results show better visual quality and less error compared with those obtained by the conventional single-shot approach on both still and moving objects.

DOI

Separation of Reflective and Fluorescent Components using the Color Mixing Matrix

Isao Shimana, Toshiyuki Amano

2017 IEEE VIRTUAL REALITY (VR) ( IEEE COMPUTER SOC )    235 - 236   2017  [Refereed]

　View Summary

In the field of SAR, the projector-camera system has been well studied; its radiometric model can be easily described by a color mixing matrix. Many SAR applications have proposed and created by using this model. However, this model can be used for reflectance component, but not for fluorescence component. In this paper, we propose RKS Projector-Camera response model for separating of the color mixing matrix's reflectance components and fluorescence components and describe how to decompose them.

DOI

Coded facial expression

Toshiyuki Amano

SA 2016 - SIGGRAPH ASIA 2016 Emerging Technologies ( Association for Computing Machinery, Inc )    11:1-11:2   2016.11  [Refereed]

DOI

Gaussian Light Field: Estimation of Viewpoint-Dependent Blurfor Optical See-Through Head-Mounted Displays

Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, Gudrun Klinker

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS ( IEEE COMPUTER SOC )  22 ( 11 ) 2368 - 2376   2016.11  [Refereed]

　View Summary

We propose a method to calibrate viewpoint-dependent, channel-wise image blur of near-eye displays. especially of Optical See-Through Head-Mounted Displays (OST-HMDs). Imperfections in HMD optics cause channel-wise image shift and blur that degrade the image quality of the display at a user's viewpoint. If we can estimate such characteristics perfectly, we could mitigate the effect by applying correction techniques from the computational photography in computer vision as analogous to cameras. Unfortunately. directly applying existing calibration techniques of cameras to OST-HMDs is not a straightforward task. Unlike ordinary imaging systems. image blur in OST-HMDs is viewpoint-dependent. i.e. the optical characteristic of a display dynamically changes depending on the current viewpoint of the user. This constraint makes the problem challenging since we must measure image blur of an HMD, ideally. over the entire 3D eyebox in which a user can see an image. To overcome this problem. we model the viewpoint-dependent blur as a Gaussian Light Field (GLF) that stores spatial information of the display screen as a (4D) light field with depth information and the blur as point-spread functions in the form of Gaussian kernels, respectively. We first describe both our GLF model and a calibration procedure to learn a GLF for a given OST-HMD. We then apply our calibration method to two HMDs that use different optics: a cubic prism or holographic gratings. The results show that our method achieves significantly better accuracy in Point-Spread Function (PSF) estimations with an accuracy about 2 to 7 dB in Peak SNR.

DOI

Appearance Manipulation for Whole Circumference by using Projector Camera Systems

島名功, 天野敏之, 牛田俊

日本バーチャルリアリティ学会論文誌 ( 日本バーチャルリアリティ学会 )  21 ( 2 ) 363‐371 - 371   2016.06  [Refereed]

　View Summary

<p>In this paper, we investigated the stability of successive omnidirectional appearance manipulation for the multiple projector camera systems without synchronization between systems. This type of system comprises several surrounding projector camera units, where each unit projects illumination independently onto a different aspect of a target object based on feedback from the projector cameras. Thus, the system can facilitate appearance manipulation from any viewpoint in the surrounding area. An advantage of this system is that it does not require information sharing or a geometrical model. However, this approach is problematic because the stability of the total control system cannot be guaranteed even if the feedback system of each projector cameras stable. Therefore, we simulated the feedback from the behavior of projector camera systems to evaluate its stability. Based on hard ware experiments, we confirmed the stability of omnidirectional appearance manipulation by using two projector camera units in an interference condition. The results showed that the object's appearance could be manipulated throughout approximately 296 degrees of the total circumference of the target object. Furthermore, we put into practice whole circumference of the object by using a plate mirror.</p>

DOI

Appearance Control in Dynamic Light Environments with a Projector-Camera System

Ryo Akiyama, Goshiro Yamamoto, Toshiyuki Amano, Takafumi Taketomi, Christian Sandor, Hirokazu Kato

2016 IEEE VR 2016 Workshop on Perceptual and Cognitive Issues in AR (PERCAR)     1 - 6   2016.03  [Refereed]

Algorithm to Compensate the Time Delay for the Appearance Control System using Camera and Projector

河野邦起, 牛田俊, 天野敏之

システム制御情報学会論文誌   29 ( 2 ) 93 - 100   2016.02  [Refereed]

　View Summary

The appearance control system is one of the visual feedback control systems that can manipulate the appearance of an object by using a projector-camera system. This system processes the captured image and projects the image on the object overlappingly for manipulating the appearance in real time. However, the afterimage is caused by a part of the image which is not overlapped on the object and remains in capture range of the camera. Therefore, this afterimage makes the appearance control system unstable. In this paper, we propose a new projection algorithm that can compensate the time delay for the appearance control system.We verify the effectiveness of our algorithm which can suppress the influence of the afterimage against the time delay in the visual feedback loop.

DOI

Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays

Yuta Itoh, Maksym Dzitsiuk, Toshiyuki Amano, Gudrun Klinker

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS ( IEEE COMPUTER SOC )  21 ( 11 ) 1269 - 1278   2015.11  [Refereed]

　View Summary

The fundamental issues in Augmented Reality (AR) are on how to naturally mediate the reality with virtual content as seen by users. In AR applications with Optical See-Through Head-Mounted Displays (OST-HMD), the issues often raise the problem of rendering color on the OST-HMD consistently to input colors. However, due to various display constraints and eye properties, it is still a challenging task to indistinguishably reproduce the colors on OST-HMDs. An approach to solve this problem is to pre-process the input color so that a user perceives the output color on the display to be the same as the input.
We propose a color calibration method for OST-HMDs. We start from modeling the physical optics in the rendering and perception process between the HMD and the eye. We treat the color distortion as a semi-parametric model which separates the non-linear color distortion and the linear color shift. We demonstrate that calibrated images regain their original appearance on two OST-HMD setups with both synthetic and real datasets. Furthermore, we analyze the limitations of the proposed method and remaining problems of the color reproduction in OST-HMDs. We then discuss how to realize more practical color reproduction methods for future HMD-eye system.

DOI

Tutorial 2: Computational Imaging and Projection.

Shinsaku Hiura, Hajime Nagahara, Daisuke Iwai, Toshiyuki Amano

2015 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2015, Fukuoka, Japan, September 29 - Oct. 3, 2015 ( IEEE Computer Society )    xxxi   2015.09  [Refereed]

DOI

Structural Color Display on Retro-reflective Objects.

Toshiyuki Amano, Kensuke Minami

International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments, ICAT-EGVE 2015, Kyoto, Japan, October 28-30, 2015. ( Eurographics Association )    37 - 44   2015  [Refereed]

DOI