Assessing stakeholder perspectives on the explainability of AI solutions for smart production planning with just-in-time logistics

• Two scenarios where smart production planning assistance improves human workflows • A questionnaire to assess levels of explainability needed to implement this system • Survey results by a group of active production planners (n=11) • Light levels of XAI are a strict requirement for an AI-based smart production system • Higher explainability is requested but should not harm performance In recent years, intelligent systems have increased their capabilities greatly increasing their practical applicability. However, for the foreseeable future, such AI-powered agents will not act autonomously but assist a human that will ultimately responsible. Here, the explainability of agents’ suggestions becomes paramount to provide trust and acceptance by their human co-workers. For the field of stochastic/evolutionary optimization, it has not yet been investigated what levels of explainability real human stakeholders without deep technical knowledge of these systems actually request. In this article, we report an exploratory case study where we questioned a group of production planners ( n = 11 ) about their needs for AI assistance and what types of explanations they would require to integrate AI into their day-to-day work-flow and still feel comfortable with the cooperation. While five participants expect their individual position to be threatened by these systems in the mid-term, all participants agree that AI is beneficial for safeguarding the location against competitors or migration. We find that AI-based assistance is requested to a large degree across all age groups and that stakeholders greatly request explainability of the agent’s recommendations. From this real-world empirical evidence it becomes evident that implementing explainable optimization in production planning is a crucial next step towards Industry 5.0.