Alumni

Yuta Itoh (伊藤 勇太), Dr. rer. nat.

Research Interests (研究)

Consistent Augmented Reality with Optical See-Through (OST) HMDs

How can we realize an AR experience which is indistinguishable (True AR) from the reality?;
especially, with Optical See-Through Head-Mounted Displays (OST-HMDs)?

Vision Augmentation with OST-HMDs

Given an ultimate OST-HMD, can we enhance our vision capablity?

Projects

Consistent Augmented Reality with Optical See-Through (OST) HMDs

• How can we realize an AR experience which is indistinguishable (True AR) from the reality?;
  especially, with Optical See-Through Head-Mounted Displays (OST-HMDs)?
  Consistency is a key requirement for ultimate OST-HMDs:
  • Spatial
    • Registering virtual objects w.r.t the real world
    • INDICA: INteraction-free DIsplay CAlibration for OST-HMD based on 3D eye localization [IEEE 3DUI'14, ISMAR'14]
    • Corneal Imaging Calibration: an eye tracking by corneal reflection of a display image[IEEE TVCG (IEEE VR '15)]
    • Correction of non-linear view-dependent optical aberration via light fields [IEEE VR'15]
      (Matlab code is available. See the software section)
    • Image and scene view aberration correction before/after [IEEE ISMAR'15]
  • Perceptual
    • Color reproduction [IEEE TVCG'(IEEE ISMAR '15)]
    • Occlusion [IEEE TVCG'(IEEE ISMAR '17)]
  • Temporal
    • Latency, i.e., a delay between the real world and virtual images to be rendered on, is a nuisance problem
    • We demonstrate an off-the-shelf, low-latency OST-HMD system built on a commercial VR HMD, i.e., the Oculus RIft DK2
• Visualization of some OST-HMD issues in AR

(The animation below works on Chrome browser only)
Your browser needs inline frame support to see this content.

Vision Augmentation with OST-HMDs

• Given an ultimate OST-HMD, can we enhance our vision capablity?
  • We conducted a conceptual experiment for defocus correction via an OST-HMD [AH 2015]
  • We could assist our visual prediction skill by combining physics prediction and OST-HMDs [AH 2016]
    We call this concept as Laplacian Vision

External Projects (外部プロジェクト)

Collective Visual Sensing, JST CREST (2016-2018)

• Understand group attention and activities by analyzing information gathered from multiple wearable devices, such as wearable cameras and eye trackers.
• This project was supported by JST CREST

(2013-2016)

• Using VR/AR for maintenance in extreme environments
• This project was supported by a Marie Curie Initial Training Network Fellowship of the European Community FP7 Programme (FP7- PEOPLE-2012-ITN) under contract number PITN-GA-2012-316919-EDUSAFE

Software (公開ソフトウェア)

OST-HMD Calibration toolbox for MATLAB

• Code available (Last update: 21.9.2014):
  • Zip (Source and dataset)
  • How to use (GitHub)
• A reference code with a real dataset
• This toolbox implements the following functions:
  • Direct Linear Transform for SPAAM (Single Point Active Alignment Method)
  • Eye position-based calibration method for INDICA (INteraction-free DIsplay CAlibration)

OST-HMD Undistortion toolbox for MATLAB

• Code available (Last update: 18.12.2015):
  • How to use (GitHub, Source and dataset)
• A reference code with a real dataset
• This toolbox implements the following functions:
  • Utility functions to convert world points into light rays in the HMD coordinate system
  • Non-parameteric Kernel regression for the light-field correction

Eye pose estimation (C++)

• Code available (Last update: 27.Oct.2016):
  • GitHub

(Deprecaated) Eye position estimation (C++)

• Code available (Last update: 31.1.2015):
  • How to use (GitHub)

Unity 3D button assets for voice commands

• Available at GitHub

Publications (業績)

Journal Papers (学術論文誌論文)

2017

Jens Gurbert, Yuta Itoh, Kenneth R. Moser, J Edward Swan II
(to appear) A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics, vol. xx, num. x pp. xxxx--xxxx.

Yuta Itoh, Takumi Hamasaki, Maki Sugimoto
Occlusion Leak Compensation for Optical See-Through Displays using a Single-layer Transmissive Spatial Light Modulator
IEEE Transactions on Visualization and Computer Graphics (will be presented at ISMAR 2017), vol. 23, num. 11 pp. 2463--2473.
(Project page)

Jason Orlosky, Yuta Itoh, Maud Ranchet, Kiyoshi Kiyokawa, John Morgan, Hannes Devos
Emulation of Physician Tasks in Eye tracked Virutal Reality for Remote Diagnosis of Neurodegenerative Disease
IEEE Transactions on Visualization and Computer Graphics (Presented at VR 2017), vol. 23, num. 4 pp. 1417--1426. (bib)

2016

Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, Gudrun Klinker
Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics (Presented at ISMAR 2016), vol. 22, num. 11 pp. 2368--2376. (bib)

2015

Yuta Itoh, Maksym Dzitsiuk, Toshiyuki Amano, Gudrun Klinker
Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics (Presented at ISMAR 2015), vol. 21, num. 11 pp. 1269--1278. (bib)

Kenneth R. Moser, Yuta Itoh, Kohei Oshima, J Edward Swan II, Gudrun Klinker, Christian Sandor
Subjective Evaluation of a Semi-Automatic Optical See-Through Head-Mounted Display Calibration Technique
IEEE Transactions on Visualization and Computer Graphics (Presented at Virtual Reality 2015), vol. 21, num. 4 pp. 491--500. (bib)

Alexander Plopski, Yuta Itoh, Christian Nitschke, Kiyoshi Kiyokawa, Gudrun Klinker, Haruo Takemura
Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics (Presented at Virtual Reality 2015), vol. 21, num. 4 pp. 481--490. (bib)

Yuta Itoh, Gudrun Klinker
Light-Field Correction for Spatial Calibration of Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics (Presented at Virtual Reality 2015), vol. 21, num. 4 pp. 471--480. (bib)

2011

Masahi Sugiyama, Taiji Suzuki, Yuta Itoh, Takafumi Kanamori, Manabu Kimura
Least-squares two-sample test
Neural Networks, vol. 24, no. 7, pp. 735--751, 2011 (bib)

Conference Papers, peer-reviewed (査読付き国際会議論文)

2017

Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, Maki Sugimoto
AdaptiVisor: Assisting Eye Adaptation via Occlusive Optical See-Through Head-Mounted Displays
The 8th Augmented Human International Conference (AH 2017), Silicon Valley, CA, USA, March 16-18, 2017, 9:1--9:9

Shigo Ko, Yuta Itoh, Yuta Sugiura, Takayuki Hoshi, Maki Sugimoto
Spatial Calibration of Airborne Ultrasound Tactile Display and Projector-Camera System Using Fur Material
Tangible and Embedded Interaction 2017, page 583-588

2016

Katsutoshi Masai, Yuta Itoh, Yuta Sugiura, Maki Sugimoto
Analysis of Multiple Users' Experience in Daily Life Using Wearable Device for Facial Expression Recognition
The ACM 13th International Conference on Advances in Computer Entertainment Technology (ACE 2016), pp 52:1--52:5, Nov. 2016.

Alexander Plopski, Jason Orlosky, Yuta Itoh, Christian Nitschke, Kiyoshi Kiyokawa, Gudrun Klinker
Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras
The 15th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2016), Merida, Mexico, Sep. 19 - 23, 2016, pp. 94--99.

Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa,Gudrun Klinker
Laplacian Vision: Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays
The 7th Augmented Human International Conference (AH 2016), Geneva, Switzerland, February 25 - 27, 2016, 16:1--16:8. Best Paper Award (bib)
Video at YouTube

2015

Yuta Itoh, Gudrun Klinker
Simultaneous Direct and Augmented View Distortion Calibration of Optical See-Through Head-Mounted Displays
The 14th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2015), Fukuoka, Japan, Sep. 29 - Oct. 3, 2015, pp. 43--48. (bib)
Video at YouTube

Yuta Itoh, Gudrun Klinker
Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays
The 6th Augmented Human International Conference (AH 2015), Singapore, March 9 - 11, 2015, pp 1-8. Best Paper Award (bib)

2014

Yuta Itoh, Gudrun Klinker
Performance and Sensitivity Analysis of INDICA : INteraction-Free DIsplay CAlibration for Optical See-Through Head-Mounted Displays
The 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014), Munich, Germany, Sep. 10 - 12, 2014, pp. 171--176. (bib)

Yuta Itoh, Gudrun Klinker
Interaction-Free Calibration for Optical See-Through Head-Mounted Displays based on 3D Eye Localization
The 9th IEEE Symposium on 3D User Interfaces (3DUI, in conjunction with IEEE Virtual Reality Conference), Minneapolis, Minnesota USA, Mar. 29 - 30, 2014, pp. 75-82. Best Paper Award (Honourable Mention) (bib)

Conference Posters, peer-reviewed (査読付き国際会議ポスター)

2017

Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, Toshiyuki Amano, Maki Sugimoto
Monocular Focus Estimation Method for a Freely-Orienting Eye using Purkinje-Sanson Images
2017 IEEE Virtual Reality, VR2017, Los Angeles, CA,USA, March 18-22, 2017, pp 213--214.

Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, Daisuke Iwai, Maki Sugimoto
HySAR : Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection
2017 IEEE Virtual Reality, VR2017, Los Angeles, CA,USA, March 18-22, 2017, pp 211--212.

2016

Katsuhiro Suzuki, Fumihiko Nakamura, Jiu Otsuka, Katsutoshi Masai, Yuta Itoh, Yuta Sugiura, Maki Sugimoto
Facial Expression Mapping inside Head Mounted Display by Embedded Optical Sensor
Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST 2016), pp 91--92, Nov. 2016.

Yuta Itoh, Jason Orlosky, Manuel Huber, Kiyoshi Kiyokawa, Gudrun Klinker
OST Rift: Temporally Consistent Augmented Reality with a Consumer Optical See-Through Head-Mounted Display
2016 IEEE Virtual Reality, VR2016, Greenville, SC,USA, March 19-23, 2016, pp 187--188. (bib)

2015

Marios Bikos, Yuta Itoh, Gudrun Klinker, Konstantinos Moustakas
An Interactive Augmented Reality Chess Game using Bare-Hand Pinch Gestures
International Conference on Cyberworlds (CW) 2015, Gotland, Sweden, October 2015, pp 355-358. (bib)

Kenneth R. Moser, Yuta Itoh, J Edward Swan II
Continuous Automatic Calibration for Optical See-through Displays
2015 IEEE Virtual Reality, VR 2015, Arles, Camargue, Provence, France, March 23-27, 2015, pp 241--242. (bib)

Conference Demos (展示等)

2016

Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker
Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors
In ACM SIGGRAPH 2016 Emerging Technologies (SIGGRAPH 2016). ACM, New York, NY, USA.

2014

Yuta Itoh, Gudrun Klinker
[DEMO] INDICA : Interaction-Free Display Calibration for Optical See-through Head-mounted Displays based on 3D Eye Localization
The 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014), Munich, Germany, Sep. 10 - 12, 2014, pp. 345--346. (bib)

2013

Yuta Itoh, Frieder Pankratz, Christian Waechter , Gudrun Klinker
[DEMO] Calibration of Head-Mounted Finger Tracking to Optical See-Through Head Mounted Display
Demonstration at the 12th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2013), Adelaide, Australia, October 1-4, 2013. (bib)

Teaching (教育)

Lectures (講義)

• SS 2014 -- Lecture (Exercises) -- Augmented Reality (IN 2018)

Running and Finished Student Projects (学生指導)

Master-, Bachelor-Theses (MA/BA), and Guided Research (GR) projects I [co]supervise[d]

• MA -- Elen Voskoboinikova -- Dynamic Correction of Lens Distortion for Optical See-Through Head-Mounted Displays -- Jan 2016 - Sep. 2016
• MA -- Μάριος Μπίκος (Marios Bikos)-- Dynamic Simulation of Virtual Objects for Augmented Reality Applications. Development of an Augmented Reality Chess -- March 2015 - May 2015
• GR -- Maksym Dzitsiuk -- Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays -- Oct 2014 - March 2015
• BA -- Maximilian Flis -- Operator Guidance at SMT Assembly Production Lines Using Google Glass -- Oct 2014 - April 2015
• BA -- Max Wandinger -- Augmented Virtual Space: Interactions in a Multi-Device Virtual Environment -- Oct 2014 - Feb 2015
• BA -- Stefan Hell -- Augmented Virtual Space: Visualizations in a Multi-Device Virtual Environment -- Oct 2014 - Feb 2015
• BA -- Maximilian Tharr -- Visualization of Relative Distance between Real and VIrtual Objects in Egocentric Augmented Reality Environments -- Jul 2014 - Feb 2015

Misc

Coordinate system conversions between UpenCV, OpenGL, and Ubitrack

Research Funds (研究助成)

• 2017/09–2022/03, PI, Precursory Research for Embryonic Science and Technology (PRESTO), JST (JST 戦略的創造研究推進事業 さきがけ)
• 2017/04–2021/03, PI, Grant-in-Aid for Young Scientists (A), JSPS KAKENHI #17H04692 (科研費 若手研究 A 代表)
• 2016/08–2017/03, PI, Grant-in-Aid for Research Activity Start-up, JSPS KAKENHI #16H07169 (科研費 研究活動スタート支援 代表)

Talks (講演)

2014

Invited Talk: Head Mounted Displays: How to realize ultimate AR experiences?
Wissens-Austausch-Workshop: The 3rd Workshop on Visualisierung großer Datenmengen in der Wissenschaft (VisDa3^? WAW), DLR@Oberpfaffenhofen, Germany, Jun. 9 - 10, 2015.
Materials at SlideShare

Tutorial: How to calibrate Optical See-Through Head Mounted Displays
Jens Gurbert and Yuta Itoh
The 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014), Munich, Germany, Sep. 10 - 12, 2014.
Materials at SlideShare

Invited Talk: Why we shouldn't use Optical-See Through Head-Mounted Displays, yet? & a PhD life in Germany
Cybermedia Center, Osaka University, Osaka, Japan, Jul. 30, Wednesday, 2014.

Academic Meetings (研究会等)

2015

Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays
Yuta Itoh and Gudrun Klinker
The Winter Augmented Reality Meeting 2015, Graz, Austria, Feb. 4 - 5, 2015.
Presentation slide

2014

Interaction-free Calibration for Optical See-through Head-Mounted Displays based on 3D Eye Localization
Yuta Itoh and Gudrun Klinker
The Winter Augmented Reality Meeting 2014, Graz, Austria, Feb. 26 - 27, 2014.

Awards (受賞)

• 2017, Augmented Human: Best Paper Award, 3rd-place
• 2016, IEEE ISMAR: Best Paper Runner Up Award
• 2016, Augmented Human: Best Paper Award
• 2015, Augmented Human (ACM In-Cooperation): Best Paper Award
• 2014, IEEE 3DUI: Best Paper Award (Honourable Mention)
• 2013, Marie Curie Actions research fellowship (ITN, EDUSAFE)