Significant advances in machine learning techniques for image classification, object detection and image segmentation have profound implications for crowdsourced mapping applications. Recent open source initiatives such as SpaceNet have strived to direct more research and development towards specific foundational mapping functions such as building detection and road network and routing identification. As these machine learning techniques mature, mapping contributors need to understand and engage the research community to help structure the application of these new techniques against a diverse of mapping challenges. Yet, currently, it is difficult translate mapping requirements to machine learning evaluation metrics, and vice versa. This presentation will discuss a proposed framework for defining levels of analyst augmentation that will allow mapping contributors and machine learning researchers to better understand each other and help direct the application of these advanced algorithms against mapping problems. Specifically, it will focus on relevant use case of mapping requirements, before, during and after a natural disaster and demonstrate a framework to understand what capabilities are nearing readiness.