气候变化领域集成服务门户(CCPortal): Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping

CCPortal

DOI	10.3390/rs16050797
	Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping
	Li, Wenwen; Hsu, Chia-Yu; Wang, Sizhe; Yang, Yezhou; Lee, Hyunho; Liljedahl, Anna; Witharana, Chandi; Yang, Yili; Rogers, Brendan M.; Arundel, Samantha T.; Jones, Matthew B.; McHenry, Kenton; Solis, Patricia
发表日期	2024
EISSN	2072-4292
起始页码	16
结束页码	5
卷号	16 期号:5
英文摘要	This paper assesses trending AI foundation models, especially emerging computer vision foundation models and their performance in natural landscape feature segmentation. While the term foundation model has quickly garnered interest from the geospatial domain, its definition remains vague. Hence, this paper will first introduce AI foundation models and their defining characteristics. Built upon the tremendous success achieved by Large Language Models (LLMs) as the foundation models for language tasks, this paper discusses the challenges of building foundation models for geospatial artificial intelligence (GeoAI) vision tasks. To evaluate the performance of large AI vision models, especially Meta's Segment Anything Model (SAM), we implemented different instance segmentation pipelines that minimize the changes to SAM to leverage its power as a foundation model. A series of prompt strategies were developed to test SAM's performance regarding its theoretical upper bound of predictive accuracy, zero-shot performance, and domain adaptability through fine-tuning. The analysis used two permafrost feature datasets, ice-wedge polygons and retrogressive thaw slumps because (1) these landform features are more challenging to segment than man-made features due to their complicated formation mechanisms, diverse forms, and vague boundaries; (2) their presence and changes are important indicators for Arctic warming and climate change. The results show that although promising, SAM still has room for improvement to support AI-augmented terrain mapping. The spatial and domain generalizability of this finding is further validated using a more general dataset EuroCrops for agricultural field mapping. Finally, we discuss future research directions that strengthen SAM's applicability in challenging geospatial domains.
英文关键词	foundation model; artificial intelligence; mapping; zero-shot; segmentation; GeoAI
语种	英语
WOS研究方向	Environmental Sciences & Ecology ; Geology ; Remote Sensing ; Imaging Science & Photographic Technology
WOS类目	Environmental Sciences ; Geosciences, Multidisciplinary ; Remote Sensing ; Imaging Science & Photographic Technology
WOS记录号	WOS:001182918300001
来源期刊	REMOTE SENSING
文献类型	期刊论文
条目标识符	http://gcip.llas.ac.cn/handle/2XKMVOVA/301102
作者单位	Arizona State University; Arizona State University-Tempe; Arizona State University; Arizona State University-Tempe; University of Connecticut; United States Department of the Interior; United States Geological Survey; National Center for Ecological Analysis & Synthesis; University of California System; University of California Santa Barbara; University of Illinois System; University of Illinois Urbana-Champaign
推荐引用方式 GB/T 7714	Li, Wenwen,Hsu, Chia-Yu,Wang, Sizhe,et al. Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping[J],2024,16(5).
APA	Li, Wenwen.,Hsu, Chia-Yu.,Wang, Sizhe.,Yang, Yezhou.,Lee, Hyunho.,...&Solis, Patricia.(2024).Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping.REMOTE SENSING,16(5).
MLA	Li, Wenwen,et al."Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping".REMOTE SENSING 16.5(2024).