Finite-State Fuzzy Constrained Markov Decision Processes for Individualized Decision Making.

Constrained Markov decision processes (CMDPs) are widely used for sequential decision-making under safety or resource limits. Fundamental limitations of existing CMDPs include binary state representations and an all-or-nothing criterion for safety determination. These limitations restrict the applicability of CMDPs in domains such as healthcare, where system states (e.g., patient conditions) are inherently uncertain or ambiguous, and safety requirements (e.g., side effects) exist along a continuum and must be individualized at each decision point. Motivated by these challenges and building on our previously developed stochastic fuzzy discrete event systems (SFDESs) theory, we develop a fuzzy CMDP framework that integrates fuzzy state representations with membership-based, per-decision safety constraints. For each subject, user-specified lower and upper membership thresholds define a safe fuzzy system state set, from which corresponding safe action sets are derived. An ideal fuzzy-state CMDP would require policy synthesis over a continuous membership space and is therefore computationally intractable. To address this, we introduce a projection that preserves the stochastic semantics and CMDP structure, in which safe fuzzy system states are projected onto binary states, yielding a tractable reduced binary CMDP representation. We show that Bellman recursion and dynamic programming remain directly applicable in the reduced CMDP, while membership-based safety constraints continue to govern policy admissibility. The projection ensures that any optimal policy in the reduced CMDP satisfies the fuzzy state safety constraints. The resulting framework unifies fuzzy state evolution and constrained stochastic optimization, enabling graded, individualized, and state-dependent safety specifications that are not expressible within conventional CMDPs based on binary or probabilistic constraints. A numerical example illustrates the framework. The fuzzy CMDP framework reduces to an SFDES-based Markov decision process (MDP) framework when no safety constraints are imposed. Together, these frameworks provide a mathematically rigorous and tractable basis for individual-specific sequential decision-making, with particular relevance in applications where intersubject variability plays a critical role, such as personalized healthcare.