YaBeSH Engineering and Technology Library

    • Journals
    • PaperQuest
    • YSE Standards
    • YaBeSH
    • Login
    View Item 
    •   YE&T Library
    • ASME
    • Journal of Autonomous Vehicles and Systems
    • View Item
    •   YE&T Library
    • ASME
    • Journal of Autonomous Vehicles and Systems
    • View Item
    • All Fields
    • Source Title
    • Year
    • Publisher
    • Title
    • Subject
    • Author
    • DOI
    • ISBN
    Advanced Search
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Archive

    Deep Visible and Thermal Camera-Based Optimal Semantic Segmentation Using Semantic Forecasting

    Source: Journal of Autonomous Vehicles and Systems:;2021:;volume( 001 ):;issue: 002::page 021006-1
    Author:
    John, Vijay
    ,
    Mita, Seiichi
    ,
    Lakshmanan, Annamalai
    ,
    Boyali, Ali
    ,
    Thompson, Simon
    DOI: 10.1115/1.4052529
    Publisher: The American Society of Mechanical Engineers (ASME)
    Abstract: Visible camera-based semantic segmentation and semantic forecasting are important perception tasks in autonomous driving. In semantic segmentation, the current frame’s pixel-level labels are estimated using the current visible frame. In semantic forecasting, the future frame’s pixel-level labels are predicted using the current and the past visible frames and pixel-level labels. While reporting state-of-the-art accuracy, both of these tasks are limited by the visible camera’s susceptibility to varying illumination, adverse weather conditions, sunlight and headlight glare, etc. In this work, we propose to address these limitations using the deep sensor fusion of the visible and the thermal camera. The proposed sensor fusion framework performs both semantic forecasting as well as an optimal semantic segmentation within a multistep iterative framework. In the first or forecasting step, the framework predicts the semantic map for the next frame. The predicted semantic map is updated in the second step, when the next visible and thermal frame is observed. The updated semantic map is considered as the optimal semantic map for the given visible-thermal frame. The semantic map forecasting and updating are iteratively performed over time. The estimated semantic maps contain the pedestrian behavior, the free space, and the pedestrian crossing labels. The pedestrian behavior is categorized based on their spatial, motion, and dynamic orientation information. The proposed framework is validated using the public KAIST dataset. A detailed comparative analysis and ablation study is performed using pixel-level classification and intersection-over-union (IOU) error metrics. The results show that the proposed framework can not only accurately forecast the semantic segmentation map but also accurately update them.
    • Download: (973.7Kb)
    • Show Full MetaData Hide Full MetaData
    • Get RIS
    • Item Order
    • Go To Publisher
    • Price: 5000 Rial
    • Statistics

      Deep Visible and Thermal Camera-Based Optimal Semantic Segmentation Using Semantic Forecasting

    URI
    http://yetl.yabesh.ir/yetl1/handle/yetl/4278404
    Collections
    • Journal of Autonomous Vehicles and Systems

    Show full item record

    contributor authorJohn, Vijay
    contributor authorMita, Seiichi
    contributor authorLakshmanan, Annamalai
    contributor authorBoyali, Ali
    contributor authorThompson, Simon
    date accessioned2022-02-06T05:37:05Z
    date available2022-02-06T05:37:05Z
    date copyright10/13/2021 12:00:00 AM
    date issued2021
    identifier issn2690-702X
    identifier otherjavs_1_2_021006.pdf
    identifier urihttp://yetl.yabesh.ir/yetl1/handle/yetl/4278404
    description abstractVisible camera-based semantic segmentation and semantic forecasting are important perception tasks in autonomous driving. In semantic segmentation, the current frame’s pixel-level labels are estimated using the current visible frame. In semantic forecasting, the future frame’s pixel-level labels are predicted using the current and the past visible frames and pixel-level labels. While reporting state-of-the-art accuracy, both of these tasks are limited by the visible camera’s susceptibility to varying illumination, adverse weather conditions, sunlight and headlight glare, etc. In this work, we propose to address these limitations using the deep sensor fusion of the visible and the thermal camera. The proposed sensor fusion framework performs both semantic forecasting as well as an optimal semantic segmentation within a multistep iterative framework. In the first or forecasting step, the framework predicts the semantic map for the next frame. The predicted semantic map is updated in the second step, when the next visible and thermal frame is observed. The updated semantic map is considered as the optimal semantic map for the given visible-thermal frame. The semantic map forecasting and updating are iteratively performed over time. The estimated semantic maps contain the pedestrian behavior, the free space, and the pedestrian crossing labels. The pedestrian behavior is categorized based on their spatial, motion, and dynamic orientation information. The proposed framework is validated using the public KAIST dataset. A detailed comparative analysis and ablation study is performed using pixel-level classification and intersection-over-union (IOU) error metrics. The results show that the proposed framework can not only accurately forecast the semantic segmentation map but also accurately update them.
    publisherThe American Society of Mechanical Engineers (ASME)
    titleDeep Visible and Thermal Camera-Based Optimal Semantic Segmentation Using Semantic Forecasting
    typeJournal Paper
    journal volume1
    journal issue2
    journal titleJournal of Autonomous Vehicles and Systems
    identifier doi10.1115/1.4052529
    journal fristpage021006-1
    journal lastpage021006-9
    page9
    treeJournal of Autonomous Vehicles and Systems:;2021:;volume( 001 ):;issue: 002
    contenttypeFulltext
    DSpace software copyright © 2002-2015  DuraSpace
    نرم افزار کتابخانه دیجیتال "دی اسپیس" فارسی شده توسط یابش برای کتابخانه های ایرانی | تماس با یابش
    yabeshDSpacePersian
     
    DSpace software copyright © 2002-2015  DuraSpace
    نرم افزار کتابخانه دیجیتال "دی اسپیس" فارسی شده توسط یابش برای کتابخانه های ایرانی | تماس با یابش
    yabeshDSpacePersian