Publications

The application of Reinforcement Learning (RL) to economic modeling reveals a fundamental conflict between the assumptions of equilibrium theory and the emergent behavior of learning agents. While canonical economic models assume atomistic agents act as “takers” of aggregate market conditions, a naive single-agent RL simulation incentivizes the agent to become a “manipulator” of its environment. We demonstrate this discrepancy within a search-and-matching model with concave production, showing that a standard RL agent learns a non-equilibrium, monopsonistic policy. Additionally, we identify a parametric bias arising from the mismatch between economic discounting and RL’s treatment of intertemporal costs. To address both issues, we propose a calibrated Mean-Field Reinforcement Learning framework that embeds a representative agent in a fixed macroeconomic field and adjusts the cost parameter to reflect economic opportunity costs. Our iterative algorithm converges to a self-consistent fixed point where the agent’s policy aligns with the competitive equilibrium. This approach provides a tractable and theoretically sound methodology for modeling learning agents in economic systems within the broader domain of computational social science.

Chen, Ruxin; Zhang, Zeqiang. (2026). "From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models." Computational Economics.
Download Paper | Website

At the laboratory scale, differentiating damage evolution mechanisms between soft and hard rocks and objectively identifying stress states are prerequisites for uncovering rock instability failure mechanisms. To comparatively analyze the differences in the failure processes of soft and hard rocks, this study systematically investigated the fracture evolution characteristics of samples of both rock types using acoustic emission (AE) parameters, and developed a machine learning-driven rock fracture state classification model. The results indicate that: soft and hard rocks exhibit significantly distinct phased characteristics in AE frequency, energy, and peak frequency across stress stages; the evolution process of internal microcracks in rocks was revealed through AE event location, average frequency (AF) and rise angle (RA) information; the evolution patterns of b-value and information entropy across “counts-spatial-energy” dimensions were analyzed, clarifying distinct failure mechanisms between soft rocks’ progressive plastic deformation and hard rocks’ abrupt brittle failure; the stress stages of soft and hard rocks were classified based on AE multi-parameters using machine learning classification methods, and the classification model showed high accuracy while quantitatively ranking feature importance. These findings not only effectively identify differences in fracture mechanisms between soft and hard rocks at the laboratory scale, but also highlight the potential of machine learning models for processing AE time-series data.

Li, Zhuang; Zhang, Zeqiang; Xu, Nuwen; Gao, Feng; Li, Biao. (2026). "Characterizing Failure Mechanism of Soft and Hard Rocks: Implication from Acoustic Emission and Machine Learning." Rock Mechanics and Rock Engineering.
Download Paper

Zhang, Zeqiang; Chen, Ruxin. (2025). "A Cognitive Perspective on Information Frictions in Labor Markets" Entropy. 27(12).
Download Paper | Website

A robust world model must strike the balance between faithfully capturing environmental dynamics and abstracting away from irrelevant content. While reconstruction-based world models ensure faithful supervision, they misallocate representational capacity by pixel area rather than dynamics relevance for visual tasks, which can cause task-irrelevant content to dominate the learned representation. Alternatively, reconstruction-free methods avoid this bias but risk discarding possibly relevant information. We propose StarWM, which uses a cross-attention module trained on self-supervised dynamics to decide where reconstruction applies. A dual-stream decoder then restricts reconstruction to the attended regions, with stop-gradient barriers preventing interference between the two objectives. These components allows reconstruction to supervise the visual content of attended regions without contaminating the latent with non-predictive information. On DeepMind Control with dynamic video backgrounds, default (reward-free) StarWM achieves the strongest performance under random-frame distractors and substantially outperforms reconstruction-based baselines under sequential video. In addition, its reward-augmented variant matches or exceeds reconstruction-free methods on sequential video, achieving the highest overall return across all distractor regimes. Mechanistic probing confirms StarWM preserves state attributes with near-perfect fidelity through long-horizon imagination while systematically discarding distractors.

Zhang, Zeqiang; Wurzberger, Fabian; Otte, Maximilian; Schmid, Daniel; Gottwald, Sebastian; Braun, Daniel. (2026). "StarWM: Self-Supervised Trained Attention Routing for Robust World Models." 40rd Annual Conference on Neural Information Processing Systems (NeurIPS). Sydney, Australia.
Download Paper

In self-supervised goal-conditioned reinforcement learning (RL) without external rewards, goals are typically specified by observations sampled from experience. However, depending on the observation structure, such a fixed representation of goals may be either too concrete (requiring exact pixel-level matches) or too abstract (involving ambiguous observations). Here we propose the construction of hierarchical latent goal spaces that integrate both concrete and abstract goals. To this end, we use an energy function to learn a partially ordered space, in which a subset relation between observations naturally induces a hierarchy from concrete to abstract goals. This representation enables agents to disambiguate specific states while also generalizing to shared concepts. In experiments on navigation and robotic manipulation, agents trained with our hierarchical goal space achieve higher task success and greater generalization to novel tasks compared to agents limited to purely observational goals.

Wurzberger, Fabian; Gottwald, Sebastian; Zhang, Zeqiang; Braun, Daniel. (2026). "Hierarchical Goal Abstractions via Learned Subset Relations." 43rd International Conference on Machine Learning (ICML). Seoul, South Korea.
Download Paper

Zhang, Zeqiang; Wurzberger, Fabian; Gottwald, Sebastian; Braun, Daniel. (2025). "Learning Robust Representations for World Models without Reward Signals." 18th European Workshop on Reinforcement Learning. Tüebingen, Germany.
Download Paper

Chen, Ruxin; Zhang, Zeqiang. (2025). "Deep Reinforcement Learning in Labor Market Simulations." 2025 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CiFer). Trondheim, Norway.
Download Paper | Website

Zhang, Zeqiang; Wurzberger, Fabian; Schmid, Gerrit; Gottwald, Sebastian; Braun, Daniel. (2025). "Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback." Arxiv.
Download Paper | Website