⚔️ The Challenge: Navigating the Unknown
Future deep space missions will push the boundaries of human-machine collaboration. Anomalies are expected, despite best efforts to prevent them.
The inevitable communication delays and limited data bandwidth in deep space will significantly reduce real-time access to Mission Control expertise. Astronauts will need to act autonomously, facing unprecedented challenges without the immediate support they’re accustomed to.
❓The Problem: Limited Support, Greater Autonomy
In deep space, the current practice of tight coordination between Mission Control and the crew will be impractical. When urgent situations arise, the crew must make critical decisions with limited guidance.
This shift demands a new approach to how monitoring and decision-making are conducted.
How communication delay can cause problems in coordination and maintaining common ground between the deep space habitat's crew and ground staff
🤔 Research Questions: Preparing for Future Missions, like Mission Mars
How can we better prepare mission control personnel and astronauts for the challenges of deep space operations?
To do this we sought to understand how Mission Control Personnel currently:
-
Maintain situation awareness
-
Make decisions
-
Respond to anomalies
📝 UX Research Methodologies
01. Data Collection: Semi-structured Interviews
23 Interviews
ISS astronauts and flight controllers. Average of 7.9 years of experience in their current roles, not including training and certification.
Platforms
Interviews:
Transcription:
Analysis:
Zoom or WebEx
Otter.ai
Dovetail
Accumulating 65+ hours
of data
Approximately 90 - 120 minutes each interview.
The interviews explored decision-making strategies like topics such as data acquisition, situation awareness, and the use of visual interfaces using cognitive task analysis techniques (CTA). The interviews were conducted remotely, recorded, and transcribed. These transcriptions were then manually reviewed for accuracy before being analyzed.
To gather insights into how participants understand & manage their tasks, acquire data, use displays, & maintain situational awareness in dynamic environment.
General Knowledge Elicitation Methods
General Knowledge Elicitation Methods
CTA technique to understand what people know and how they know it: skill, strategy, and judgement, that underlie performance.
Why this method? 👆🏼➡️
Critical Decision Method
(CDM)
To dive deeper into specific, high-stakes decisions, allowing us to understand the cognitive processes behind crucial actions, especially during anomalies.
Critical Decision Method
(CDM)
CTA technique to gather information about how experts make decisions in complex situations.
Why this method? 👆🏼➡️
Snowball Sampling
Given the specialized nature of the field, it was effective in reaching individuals with the specific expertise. With a limited timeline, it allowed to quickly connect with qualified participants via referrals.
Participant Recruitment
Participants with experience in ISS or shuttle mission control were recruited via internal NASA communications and through snowball sampling.
Why Snowball Sampling? 👆🏼➡️
02. Data Analysis: Hybrid Thematic Analysis
We employed a hybrid thematic analysis approach, combining inductive and deductive coding strategies. This mixed approach allowed us to balance the discovery of new insights with the validation of existing theories, providing a comprehensive understanding of the cognitive challenges faced by mission control personnel and astronauts.
Inductive Coding
(Bottom-up)
Independently identified patterns and themes emerging directly from data, to ensure analysis remained grounded in participant's actual experiences.
Deductive Coding
(Top-down)
Guided by existing literature on cognitive engineering and naturalistic decision-making, priori codes to explore known challenges were applied.
Screenshot showing codes applied for a section of the transcript for one participant
03. Validation: Data Triangulation
Data triangulation allowed us to cross-verify data, minimize biases, and enhance the credibility of our conclusions.
Through this approach, we confirmed the consistency of key themes, addressed discrepancies, and ensured that our conclusions accurately reflected the complexities of space mission operations.
It’s organized chaos - you've people on console who are doing the immediate plan [...] and then somebody in the backroom might take it and go off and work with those specialists, who are scrambling to figure out what to do to incorporate as much as they can to hand it back over to the front team to review. [...] you kind of break it up so that the areas of chaos are managed appropriately.
- FAO controller
01. Collaborative Efforts: Decision Making is Highly Distributed
🪐 So what does this imply for deep space missions?
Communication delays will make distributed decision-making more challenging; there is a clear need for automation. However, this automation must account for the tacit knowledge and accumulated expertise that human teams bring. Automated systems will need to integrate and simulate this nuanced human expertise to effectively support decision-making in deep space.
On the ISS, anomaly response and decision-making are not isolated tasks but highly collaborative efforts. Flight controllers work together, sharing information and making decisions as a coordinated team. This distributed approach helps establish common ground - shared assumptions, awareness, beliefs, and strategies - and provides valuable checks and balances through collective expertise.
☄️ Impact of the Research
Shaping Future Space Exploration Systems
Our study offers vital insights from ISS mission control practices, establishing a framework for designing resilient human-machine systems for future space exploration. A detailed report has been submitted to NASA to inform their system development.
Empirical Validation via Human Experiments
We conducted a human experiment to evaluate the effects of predictive telemetry on decision-making and task performance in high-stakes environments. The research aims to quantify the benefits of predictive visualization, guiding the development of effective telemetry systems for safer deep space operations.
These outcomes demonstrate the practical application of our research, bridging the gap between current practices and future needs, advancing human-machine collaboration in challenging space environments.
Screenshot of Boundary lines (in red) on ISS mission control displays
🪐 What are the Implications in Deep Space?
Replicating these nuanced, individualized monitoring strategies in automated systems is challenging. While automation can flag threshold breaches, it often lacks the contextual understanding that human controllers bring. Context determines meaning, wrapped up in an interconnected web of expectations and various forms of explicit and tacit knowledge. The challenge lies in designing automated systems that can interpret and prioritize data with the same level of expertise, avoid false alarms, and unnecessary interventions.
02. Anomaly Response = Monitoring Techniques + Context + Tacit Knowledge
How do flight controllers determine what to pay attention to amidst the many potentially important telemetry signals? They use a blend of active and passive monitoring practices to detect and respond to anomalies.
👀 Active Monitoring with Scan Patterns
Flight controllers use ‘scan patterns’ to actively monitor their systems, a process that is highly individualized. As one controller put it:
We flight controllers call it a scan pattern where if your head’s down looking at a procedure for a while, and you’re not looking at the data, once you do pop your head up to look through your displays, you look at your plots first to make sure nothing’s changed.
- ADCO controller
📈 Passive Monitoring with Limit or Boundary Lines
Limit lines’ serve as thresholds that indicate potential issues, but crossing a threshold alone doesn’t always trigger action. Controllers look for persistence and the rate of change to decide whether intervention is needed.
As we prepare for deep space missions, AI and automation will be essential but cannot fully replace human expertise due to several critical factors:
-
Tacit knowledge is difficult to replicate
-
Contextual understanding is vital for interpreting data effectively
-
Predictive modeling must integrate with human judgement
Ultimately, the focus should be on developing hybrid systems that enhance, rather than replace, human decision-making ensuring resilience and adaptability in future exploration.
🔍 Findings: Role of Human Expertise and Potential for Automation
Even in a well-established system with decades of operations like the ISS, anomalies are inevitable. Flight controllers are extensively trained in procedures and flight rules designed to guide actions during disruptions. However, plans are temporary and these guidelines often fall short in the face of unanticipated challenges, requiring controllers to think critically and adapt on the fly.
This raises an important question:
As we look towards deep space missions, where communication delays will make reliance on ground support impractical, could AI and automation be the answer to handling such challenges?
🪐 What are the Implications in Deep Space Missions?
For predictive models to be effective in dynamic environment, they must have a deep understanding of the context in which they are used. The models must be embedded in the current situation, knowing what has been done, what is left to do, and what roadblocks may still be encountered. They must be carefully integrated with human expertise to ensure they complement, rather than replace the nuanced decision-making required in anomaly response.
03. Predictive Modeling: The Value of Looking Ahead
Predictive modeling allows controllers to ‘look ahead,’ providing a range of benefits for the cognitive work involved in anomaly response, including hypothesis generation, diagnosis, ‘safing’, and planning.
For certain types of data, predictive capabilities are already in use. However, not all telemetry data are deterministic - some are subject to high uncertainty and rapid change.
Designing for Resilient Extra Terrestrial Habitats
Investigated how Mission Control personnel maintain situational awareness, make decisions, and respond to anomalies.
Explored how these practices and the use of AI and automation could help future deep space missions, such as those on Mars.
👀 Sneak Peak: Snapshots of some Redesigned Open MCT Displays
In the human subject experiment (more details below), participants were tasked with using our Open Mission Control Technology (MCT) based visual interface to complete objectives during a simulated scenario. The images below showcase redesigned Open MCT screens that incorporate insights from our research and were used for empirical validation in the human subject experiment.
🏛️ ORGANIZATION
-
DVC Lab x RETH Institute
-
NASA
👩🏻💻 ROLE
-
Researcher (Team of 5 UXers)
-
January '23 - May '24
📝 RESEARCH METHODOLOGIES
01. Semi-structured interviews
-
Cognitive task analysis
-
General knowledge Elicitation
-
Critical Decision Method
-
02. Thematic analysis
03. Data Triangulation
🧗🏻♀️ Constraints and Overcoming them
The team, comprising 4 researchers from engineering and HCI backgrounds, had limited prior experience in space operations, which posed challenges in fully grasping the intricacies of the field.
To mitigate these limitations, we sought guidance from experienced professionals in the space industry. By engaging with managers, engineers, and leaders from both the private and public sectors, we were able to calibrate our approach and refine our study protocol to better align with the domain's specific needs.