Chapter Four · failure evidence
What Markov Models got wrong, from 68 dissertations
The records document numerous empirical and algorithmic failures of Markov models across Markov decision processes, hidden Markov models, and Markov chain Monte Carlo methods. Practitioners frequently encountered computational bottlenecks from state space explosion, invalid memoryless assumptions, and severe parameter estimation difficulties under limited or noisy data. These records come from PhD theses at 25 institutions, 2021 to 2026. Each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
State space explosion and computational intractability limit model scalability
Practitioners rejected or abandoned Markov formulations because the exponential growth of state spaces made exact solving and parameter estimation computationally prohibitive. Summing over full path trajectories or computing generator matrix exponentials became intractable across large networks, extended lead times, and multi-queue systems.
Considered and rejected
Considered and rejected: Rejected recursive Hidden Markov Model (HMM) parameter estimation due to high computational expense scaling with domain/model space, replacing it with deep dynamic autoencoders with replay buffers.
Cooperative and Distributed Algorithms for Dynamic Fire Coverage using a Team of UAVs · unevada
Lost to a baseline
Markov networks and Markov random fields could not be used due to computational infeasibility with large species numbers and errors with common/rare species, defaulting to simpler GLM and correlation methods.
Responses of ecological communities to deforestation: an empirical and theoretical perspective · Imperial
Considered and rejected
Considered and rejected: Rejected treating full sequence dynamics as an explicit Markov chain path trajectory because summing over all intermediate sequence paths is computationally intractable.
Unsupervised inference methods for protein sequence data · IRIS - POLITO - prod
Considered and rejected
Considered and rejected: Rejected naive full state space exploration in allotetraploid HMM due to exponential computational complexity; constrained state space using a first-order Markov chain on covering reads.
Unsupervised learning with high-throughput sequencing data · Iowa State
Considered and rejected
Considered and rejected: Markov-Chain Monte Carlo (MCMC) sampling for Bayesian harmonic inference; rejected because it became computationally intractable when scaling across spatially coherent networks of thousands of nodes.
Spatiotemporal tidal prediction and analysis through physics-informed machine learning · Oxford
Considered and rejected
Considered and rejected: Rejected direct parameterization and likelihood evaluation of the CTMC generator L due to intractability, numerical instability of matrix exponentials, and the Markov chain embedding problem.
Considered and rejected
Considered and rejected: Rejected standard Markov Processes because they require constant transition rates (exponential distributions) and suffer from state-space explosion.
Risk modelling of safety critical systems for life extension in offshore oil and gas · University of Nottingham Repository
Considered and rejected
Considered and rejected: Rejected Markov models because of constant-rate memoryless assumptions and state space explosion when modeling complex interacting asset sub-processes
Renewals scheduling for high-speed railway assets · University of Nottingham Repository
Considered and rejected
Considered and rejected: Rejected computing VaR exactly via the worst possible history in the continuous state space Markov chain, choosing Monte Carlo estimation instead due to computational intractability.
Risk-sensitive and robust model-based reinforcement learning and planning · Oxford
Tried and failed
multi-armed bandit over full policy space applied to Markov decision process planning. Outcome: infeasible cost. Reason: Exponential regret in state space and horizon from ignoring correlations between overlapping policies.
Considered and rejected
Considered and rejected: Rejected full Markov Decision Process (MDP) formulation for contested logistics because solving it is currently computationally intractable or requires oversimplification.
GAME-THEORETIC MODELS FOR RAPID OPERATIONAL AIRLIFT NETWORK DESIGN IN CONTESTED ENVIRONMENTS · Calhoun
Considered and rejected
Considered and rejected: Rejected standard discrete probabilistic Markov Decision Processes (MDPs) for multi-queue network control due to the curse of dimensionality and inability to handle non-exponential service distributions or propagation time-delays.
Modeling and Control of Queuing Networks: Applications to Airport Surface Operations · MIT
Considered and rejected
Considered and rejected: Rejected using Markov Decision Process (MDP) for dynamic pricing due to exponential state space explosion with network size (|V|, |A|) and lead times (L), choosing a fluid approximation model instead.
Marketplace Design for Crowdsourced Same-Day Delivery · Georgia Tech
Memoryless assumptions fail to represent history and duration dependence
Standard Markov models failed when system transitions depended on sojourn times, wear history, or non-exponential durations rather than memoryless properties. Imposing Markov assumptions broke long-range sequence constraints, prevented modeling parallel tasks, and degraded temporal tracking accuracy.
Tried and failed
recursive discrete Bayes filters for sequence recognition applied to multimodal verbal and gesture command recognition. Outcome: worse than baseline. Reason: Markov assumption failed for temporal sequences and compounding multi-step pipeline errors degraded system-level accuracy
Tried and failed
non-homogeneous hidden semi-Markov models applied to early-stage component degradation tracking. Outcome: worse than baseline. Reason: statistical assumptions failed to capture ground truth wear dynamics during early operational life
Digital Twin-Driven Condition Monitoring Approach for Aircraft Carbon Brakes · Georgia Tech
Considered and rejected
Considered and rejected: Rejected Markov model for baseline hazard in favor of semi-Markov conditional hazard h03(t2 - t1) to capture sojourn time dependence.
Evaluation of multivariate longitudinal data accounting for missingness: methods and applications · OpenBU
Considered and rejected
Considered and rejected: Rejected direct expectation constraints on long-range sequence disjunctions (e.g., requiring at least one noun anywhere after 'the' with high probability) because it breaks the Markov factorization.
Posterior Regularization for Learning with Side Information and Weak Supervision · Penn
Considered and rejected
Considered and rejected: Rejected simulating bad data injection attack propagation using only pure Markov chains because serial node chains cannot properly express parallel tasks and cannot accommodate vastly different time resolutions between preparation (months/years) and execution (hours).
CYBER THREAT PROPAGATION MODELING IN CYBER PHYSICAL SYSTEMS · Georgia Tech
Considered and rejected
Considered and rejected: Rejected Markov chains / continuous-time Markov models because the memoryless exponential distribution cannot represent Weibull-distributed track degradation or multi-action maintenance history.
Railway track asset management modelling · University of Nottingham Repository
Considered and rejected
Considered and rejected: Rejected extracting multiple hitting-time samples from repeated node visits in a single random walk trajectory because it introduces statistical dependency and invalidates Markov properties.
New Directions in Bandit Learning: Singularities and Random Walk Feedback · DukeSpace
Considered and rejected
Considered and rejected: Rejected standard first-order Markov chain models for frailty progression in favor of semi-Markov models because transition hazards depend on sojourn time in the current and previous states.
Considered and rejected
Considered and rejected: Rejected Hidden Markov Models (HMMs) because they overlook long-term dependencies and lack explicit parameter uncertainty modeling at urban scales.
Evaluating the effects of occupancy on energy use and indoor environmental quality in residential building archetypes over spatial and temporal scales · Research Repository UCD
Considered and rejected
Considered and rejected: Rejected using duration-dependent transition probabilities in the natural history model because memoryless Markov property limited modeling duration directly, using age-specific proxies instead.
Identifying opportunities to reduce cervical cancer prevention disparities in Western Washington · ResearchWorks
Considered and rejected
Considered and rejected: Rejected using Multi-Armed Bandit search arms directly as RL actions because population-based search breaks the Markov property.
ELECTRONIC DESIGN AUTOMATION FOR HIGH-PERFORMANCE AND RELIABLE 3D MEMORY CUBES AND PROCESSORS · Georgia Tech
Data sparsity and overparameterization degrade hidden Markov model estimation
Hidden Markov models suffered when training datasets lacked sufficient observation frequencies, contained rare transitions, or had excessive noise. Adding extra observable features or state counts caused over-parameterization, unidentifiable states, and degraded prediction accuracy on short sequences.
Tried and failed
Hidden Markov Model with extra observable features applied to latent behavioural state decoding. Outcome: worse than baseline. Reason: Adding a third observable degraded prediction of active wake states compared to a two-observable model
Investigating behavioural correlates of sleep states in drosophila · Imperial
Tried and failed
two-stage pipeline training of hybrid sequence model applied to discriminative sequence modeling with hidden markov models. Outcome: worse than baseline. Reason: fixed intermediate outputs prevented joint optimization, drastically hurting test negative log-likelihood compared to end-to-end training
Tried and failed
Gaussian Hidden Markov Models with Viterbi decoding applied to hidden state estimation in time-series trajectories. Outcome: data insufficient. Reason: Lack of realistic transition matrices and extremely rare state transitions (<1%) in sequential data
Development of A Trajectory Population Data and its Application in CAV Research · Virginia Tech
Tried and failed
PCA dimensionality reduction on temporal sequence features applied to hidden Markov model state prediction. Outcome: worse than baseline. Reason: Component reduction discarded informative variance and dynamic correlations present in raw sequential features
Development of A Trajectory Population Data and its Application in CAV Research · Virginia Tech
Tried and failed
unstructured hidden Markov models with high state counts applied to continuous movement trait evolution. Outcome: overfit. Reason: over-parameterization caused model rejection and made transition rate interpretation difficult
When the Map Fails the Territory: Hidden State Models, Complex Traits and the Evolution of Bird Migration · Virginia Tech
Considered and rejected
Considered and rejected: Rejected using Hidden Markov Models (HMMs) for behavioral state classification (travel vs stationary) due to insufficiently high GPS fix frequencies (>1 min).
Factors Associated with Space Use and Reproductive Success of American Kestrels in Northern Virginia, USA · Virginia Tech
Considered and rejected
Considered and rejected: Rejected data-driven Hidden Markov Model (HMM) stride detection due to poor performance on short walking bouts (<30 strides), which make up ~65% of daily strides.
Real world multimodal assessment of physical behavior change in health and disease · EPFL
Considered and rejected
Considered and rejected: Hidden Markov models were abandoned in favor of Bayesian multivariate logistic regression due to state identifiability issues and difficulty incorporating multiple time-varying covariates.
Audience Reach Projection for the 2010 FIFA World Cup in South Africa · Penn
Considered and rejected
Considered and rejected: Rejected Hidden Markov Models (HMM) for estimating unobserved HCV infection and reinfection risk trajectories because HMMs require a priori specification of transition matrices and functional relationships with covariates.
Implications of reinfection for Hepatitis C elimination among people who inject drugs · JScholarship
Considered and rejected
Considered and rejected: Full Markov Decision Process (MDP) modeling rejected for contextual bandits with surrogate reward costs due to severe state space constraints, high noise, and sample sparsity causing inaccurate transition models.
Deployable Online Reinforcement Learning Algorithms · Harvard
Considered and rejected
Considered and rejected: Rejected full Markov Decision Process (MDP) modeling for delayed effects due to limited data and noise, opting instead for engineered bandit rewards.
Deployable Online Reinforcement Learning Algorithms for Use-inspired Research in Digital Health · Harvard
Mathematical formulation flaws and optimization pathologies destabilize policy synthesis
Formulating decision tasks as Markov decision processes failed due to intractable marginal regularity constraints, invalid stationary joint distributions, and high estimator variance. Optimization procedures also converged to sub-optimal local extrema, degenerate pure strategies, or incentive structures where agents indefinitely delayed reaching target states.
Tried and failed
Marginal polytope constraints for joint distribution validity applied to partially observed stationary Markov decision processes. Reason: Marginal state-action-state constraints fail to guarantee existence of a valid underlying stationary joint transition distribution.
Towards Credible and Effective Data-Driven Decision-Making · Cornell
Tried and failed
unconstrained conditional maximum likelihood estimation applied to linear Markov decision process representation learning. Reason: intractable marginal regularity constraints requiring integrals over continuous state spaces to sum to one
Towards provable and practical reinforcement learning with representation learning · UT Austin
Tried and failed
Markov decision process linear programming formulation applied to individual asset maintenance scheduling. Reason: optimises aggregate state proportions rather than assigning specific actions to discrete individual entities
Tried and failed
deterministic Markovian policy modifications applied to constrained correlated equilibrium search. Outcome: worse than baseline. Reason: yields a strictly weaker equilibrium concept than allowing stochastic modifications in constrained games
Game-Theoretic Decision Making in Multi-Agent Systems under Constraints and Welfare Objectives · EPFL
Tried and failed
unbiased loss estimation in policy search applied to adversarial episodic markov decision processes. Outcome: unstable. Reason: high variance causes random regret to be linear with nonzero probability
Efficient, reliable, and interpretable decision-making for human-autonomy co-existence · UT Austin
Tried and failed
sequential convex programming with trust regions applied to parametric Markov decision process synthesis. Reason: algorithm converged to a local optimum exceeding the required threshold specification
Tried and failed
game-theoretic Markov game with opponent awareness modeling applied to sequential competitive action selection. Reason: Degenerated into a deterministic pure strategy instead of a mixed policy under asymmetric information.
Optimal Pitch Selection Policies Via Markov Decision Processes · Harvard
Tried and failed
infinite-horizon augmentation without padding applied to dialogue Markov decision processes. Reason: policy values depend on variable episode turn lengths
Reducing Human Labor Cost in Deep Learning for Natural Language Processing · Georgia Tech
Tried and failed
marginalized importance sampling with nonparametric estimators applied to off-policy evaluation in Markov decision processes. Reason: the estimator does not represent a valid influence function, failing to achieve the semiparametric efficiency bound
Tried and failed
Discounted cost minimization with reachability constraints applied to Markov decision processes. Reason: Agents delay target reaching indefinitely to drive discounted costs to zero, so optimal policies do not exist.
Algorithms for cooperative and competitive autonomous systems · UT Austin
Markov models fail to capture complex nonlinear and nonstationary dynamics
Discrete transition structures were unable to model sudden non-stationary reliability shocks, complex feature interactions, and unobserved rate heterogeneity. Simplifying assumptions such as mean-field approximations or constrained directional rates led to excessive false positive rates and missed critical failure states.
Tried and failed
training separate time-period-specific hidden Markov models applied to temporal sequence prediction. Outcome: worse than baseline. Reason: a single unified model performs as well as or better than separate time-sliced models
Disaggregated short-term travel location prediction · Imperial
Tried and failed
multitype branching process approximation of Markov chains applied to periodic extinction probability boundary estimation. Outcome: did not generalise. Reason: fails to predict dominance boundaries when periodic parameters across compartments have phase shifts or asynchrony
Tried and failed
Markov chain model applied to infrastructure degradation state prediction. Outcome: worse than baseline. Reason: Inability to capture complex nonlinear dependencies and feature interactions compared to supervised machine learning models.
Tried and failed
low-order discrete Markov chain applied to weather dynamics simulation. Reason: state space was too simple to capture complex temporal fluctuations
Large scale data analytics for resilience of energy networks · Georgia Tech
Tried and failed
discrete Hidden Markov Model sequential state prediction applied to human compliance dynamics under system errors. Outcome: did not generalise. Reason: model fails to capture sudden non-stationary state shifts caused by negative reliability shocks
Human-Robot Trust in Time-Sensitive Scenarios · Georgia Tech
Tried and failed
mean-field Markov chain approximations applied to spatial lattice excitation dynamics. Outcome: did not generalise. Reason: mean-field approximations omit critical multi-site spatial defect structures that drive persistent failure states
Tried and failed
standard Markov models of discrete character evolution applied to complex discrete trait evolutionary transitions. Outcome: worse than baseline. Reason: substantially worse fit than hidden-state models due to unobserved rate heterogeneity across lineages
When the Map Fails the Territory: Hidden State Models, Complex Traits and the Evolution of Bird Migration · Virginia Tech
Tried and failed
constrained continuous-time Markov transition model applied to discrete morphological trait evolution. Outcome: worse than baseline. Reason: constrained directional transition rate assumptions fit empirical evolutionary data substantially worse than standard symmetrical rates
Tried and failed
hidden Markov model for anomaly detection applied to combustion instability detection. Outcome: did not generalise. Reason: excessive false positive rate classifying normal conditions as anomalies
Trustworthy deep learning for cyber-physical systems · Iowa State
Tried and failed
applying nominal policies to uncertain environments applied to partially observable Markov decision processes. Outcome: did not generalise. Reason: nominal control policies lack robustness to parametric model uncertainties, drastically increasing failure probability
Markov chain sampling algorithms suffer from nonergodicity and poor mixing
Markov chain Monte Carlo methods failed when induced chains were non-ergodic or suffered from diverging total variation sums. In addition, random parameter orderings and worker interruption biases disrupted stationarity or caused random walks to mix too slowly to converge.
Tried and failed
Gibbs sampling applied to linear Gaussian random acceleration systems. Outcome: did not converge. Reason: The Markov chain induced by Gibbs updates is non-ergodic for random acceleration dynamics.
Uncertainty Quantification and Structure Discovery for Scalable Behavior Science · MIT
Tried and failed
complex Markov chain dynamics applied to causal set growth models. Outcome: did not converge. Reason: divergence of total variation sum prevents well-defined decoherence functional on the full measure algebra
Discrete random spacetimes: covariance and quantization in growth dynamics for causal sets · Imperial
Considered and rejected
Considered and rejected: Markov Chain Monte Carlo (MCMC) targeting the posterior was rejected for dynamic sequence tracking due to the requirement to resample entire trajectories rather than updating sequentially.
Mapping dynamic brain networks with MEG data using machine learning · Oxford
Tried and failed
randomizing sparse approximation orderings between MCMC iterations applied to hierarchical Gaussian process inference. Outcome: unstable. Reason: changing orderings between iterations disrupted Markov chain stationarity and prevented burn-in
Deep Gaussian Process Surrogates for Computer Experiments · Virginia Tech
Tried and failed
real-time interruption replica exchange without worker filtering applied to parallel Markov chain Monte Carlo. Reason: sampling actively running chains introduces length bias, converging to erroneous shifted target distributions
Monte Carlo Methods in Practice and Efficiency Enhancements via Parallel Computation · Cambridge
Considered and rejected
Considered and rejected: Rejected the Flip chain (single-boundary-cell reassignment Markov chain) because random walks mix extremely slowly and consistently produce non-compact districts.
Algorithms for Fair Redistricting · Harvard
Considered and rejected
Considered and rejected: Setting swap rate p = 0 (attempting swaps with zero parallel moves between them) was rejected because it only permutes chain configurations without advancing Markov chain states.
Monte Carlo Simulations of Strand Passage in Unknotted Self-Avoiding Polygons · HARVEST
Tried and failed
sequential convex programming and off-the-shelf QCQP solvers applied to large parametric Markov decision processes. Outcome: did not converge. Reason: conservative step sizing and numerical instability caused timeouts on large models
Left open by the authors
Problems the authors named and did not get to.
Left open
Implement and evaluate split-merge algorithms for dynamic state space scaling in Hidden Markov Models for NLP tasks. Blocker: None
Structured State Tracking for Natural Language Understanding · Cornell
Left open
Extend state-based block Hidden Markov Models with shared transition parameters to more complex reinforcement learning behavioral tasks. Blocker: None
Behavioral Strategies and Neural Mechanisms of Dynamic Foraging · MIT
Left open
Generalize the distributed estimation heterogeneity framework to handle non-i.i.d. Markov chain data models. Blocker: Theoretical definition and estimators for Markov chain heterogeneity must be developed from scratch without a stated mathematical approach.
Data Heterogeneity in Linear Distributed Estimation and Learning · EPFL
Left open
Model correlated edge failures across network graphs using Markov chains instead of assuming independent edge failures. Blocker: None
Unveiling Roadway Network Safety: Application of Statistical Physics to Crowdsourced Velocity Data · MIT
Left open
Scale multivariate Markov chain marginal parametrisation inference to high-dimensional state spaces and large numbers of chains. Blocker: The unfinished work is an open-ended research direction without a concrete proposed algorithm or specification.
Parametrisations for inference on the dependency structures in Multivariate Markov Chains · University of Nottingham Repository
Left open
Model high-order and long-term sequential dependencies in Markov chains for recommendation while mitigating state space explosion. Blocker: No specific approach, mathematical formulation, or mitigation technique is defined to handle the state explosion
Semantic Embedded Sequential Recommendation for E-Commerce Products through Mining Customers’ Historical Interactions and Products’ Data · Scholarship at UWindsor Institutional Repository
Left open
Construct a formal statistical description and Markov partition of turbulence directly from exact coherent structure networks. Blocker: Lacks specific algorithmic methodology for partitioning continuous high-dimensional phase space into symbolic Markov states
On the Generalization of Shadowing to Fluid Turbulence: Practical Methods For Quantifying Dynamical Similarity · Georgia Tech
Left open
Develop automated structure learning and model selection algorithms for multivariate Markov chain dependency graphs. Blocker: No specific objective function, search strategy, or concrete framework is specified beyond a general goal
Parametrisations for inference on the dependency structures in Multivariate Markov Chains · University of Nottingham Repository
Left open
Derive the random-order universal prediction regret and optimal prediction strategy for non-sequential generative models trained on Markov sources. Blocker: None
Universal Prediction in the Age of Large Language Models · EPFL
Left open
Develop probabilistic inference algorithms using the proposed Hierarchical Markov Networks. Blocker: No concrete algorithmic approach or target inference task is specified
A knowledge representation scheme for the Bayesian network model. · oURspace
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.