IEEE CiFer 2025 · Symposium on Computational Intelligence for Financial Engineering and Economics

Deep Reinforcement Learning in Labor Market Simulations

Ruxin Chen
Department of Economics
Nagoya University, Japan
Zeqiang Zhang
Institute of Neural Information Processing
Ulm University, Germany

RL-EM · the Reinforcement-Learning-based Employment Model

Abstract

This paper proposes a framework for applying reinforcement learning (RL) within agent-based models (ABMs) to study labor market dynamics. ABMs provide a flexible platform for simulating economic systems, particularly by modelling heterogeneous agents and their complex interactions, which effectively capture non-linear and emergent phenomena in labor markets. We extend an existing labor market model by integrating it into an RL framework, using the Deep Deterministic Policy Gradient (DDPG) algorithm to train agents to maximize profits, and comparing their performance with bounded-rational agents governed by predefined policies. Our findings show that RL agents, depending on the level of competition and rationality in the market, spontaneously learn distinct strategies, which significantly impact outcomes such as unemployment and wage distribution.

Motivation

An agent-based model must be told how its agents decide

Macroeconomics has long relied on models that assume what firms want and how they get it. Agent-based models relax the equilibrium assumptions but replace them with a different commitment: every agent needs a decision rule, written down in advance. This paper removes that commitment for one class of agent and asks what the agent does instead.

Dynamic stochastic general equilibrium

Assumed behavior

DSGE models are used extensively to analyze policy and shocks, but rely on strong assumptions about agent behavior and market conditions, which limits their realism and their applicability to real-world scenarios.

Agent-based models

Prescribed behavior

ABMs let macroeconomic phenomena emerge from bottom-up interactions of heterogeneous agents, without assuming equilibrium. But the decision rules must still be predefined, which restricts the model's ability to adapt to new conditions or learn from experience.

This paper · RL-EM

Discovered behavior

A firm's policy is learned by trial-and-error interaction with the market rather than written down. What the firm ends up doing is then an observation about the market, not an assumption about the firm.

The gap this addresses. RL has been applied to firm and consumer behavior in microeconomics, and to Real Business Cycle models in macroeconomics. Its use to model labor market dynamics has remained largely unexplored: existing agent-based labor market studies use traditional, rule-based ABMs. To the authors' knowledge, this is the first labor market modelled with RL agents.
The RL-EM Model

A Diamond–Mortensen–Pissarides market with a learning firm in it

Workers flow between unemployment and employment. Firms post vacancies and wages; a matching function converts the two stocks into hires; job seekers pick the best-paying option among the nf they happen to see. The only thing that changes between the baseline and RL-EM is who decides the vacancies and wages.

The RL-EM model: labor market flows on the left, the matching process in the middle, and the two firm types on the right.
Fig. 1 · The RL-EM model. Left: labor market flows — employed workers become unemployed with separation probability λ, and unemployed workers re-enter through the matching process. Middle: at the beginning of each step every firm sets its vacancies and wage; the number of matches follows the matching function; successfully matched seekers choose the highest-wage job among nf randomly drawn alternatives. Right: the two firm types — an RL firm that learns its policy with DDPG to maximize cumulative reward, and a trend follower with a fixed policy that follows the market trend. Reproduced from the paper.

Wages are set through one number

A firm does not name a wage directly. It chooses a wage determination factor b that interpolates between the unemployment benefit z and the added value of a worker p. All of the firm's bargaining posture is compressed into that single scalar.

(1) w=bz+ (1−b)p

With z=500 and p=1000, b=1 pays the reservation wage and b=0 hands the entire surplus to the worker.

Hires come from a matching function

The market is not a market-clearing auction. The number of hires in a step is a Cobb–Douglas function of the two aggregate stocks: unemployed workers u and posted vacancies v. Employment and unemployment then follow a simple stock adjustment.

m(u,v)= Auα v1−α

A=0.471, α=0.6, both taken from the empirical labor literature; separations occur at rate λ=0.1.

Firm type 1 · the learner

The RL firm

Observes the market state after the previous step — unemployment, its own headcount, total vacancies, total and own matches, the average and its own wage — and emits an action (v, b). It is trained with DDPG to maximize discounted profit, receiving a penalty of −100 whenever it ends a step with no employees.

π=arg maxπ 𝔼π[ ∑t=0T γtrt]

Actor: 4 fully connected layers. Critic: 3 layers, with a target network and Ornstein–Uhlenbeck exploration noise. Curriculum learning introduces the agent after 5 steps.

Firm type 2 · the baseline

The trend follower

A bounded-rational rule with no objective function. Each step it compares its wage against the market average (wage delta ΔW) and its realized hires against what its vacancy share would have earned at the market fill rate (expected vacancies delta ΔE), then adjusts one of the two levers — Eq. (5) and (6) of the paper.

SignalReadingResponse
ΔW<0, ΔE<0underpaying, under-hiringraise the wage
ΔW>0, ΔE>0overpaying, over-hiringcut the wage
ΔW<0, ΔE>0cheap and effectivepost more vacancies
ΔW>0, ΔE<0expensive and ineffectivepost fewer vacancies
Experimental parameters (Table I)
SymbolDescriptionValue
zUnemployment benefit500
pAdded value of a worker1000
cRecruitment cost per vacancy100
AMatching function parameter0.471
αMatching elasticity0.6
λDismissal rate0.1
TNumber of steps500
NFirmNumber of firms5
NWorkerInitial unemployed workers300
NWorker/firmInitial employed workers per firm5
binitInitial wage determination factor0.5
μbTrend-follower wage adjustment rate0.5
μvTrend-follower vacancy adjustment rate0.1
γDiscount factor0.9
lractorActor learning rate10⁻⁴
lrcriticCritic learning rate10⁻³

Input states are normalized to [0,1] and profits and rewards are divided by 10⁴ for stability. All reward figures on this page use that same scaling.

The competition parameter

nf is the whole experiment

A matched job seeker draws nf vacancies at random and takes the highest-paying one. From the firm's side, nf is how much competition it faces; from the worker's side, it is how transparent the market is. Every result in the paper is a sweep over this one number.

Interactive

Who gets the applicants?

Five firms post vacancies at different wages. The paper states the selection rule in one sentence — “each unemployed worker randomly selects nf jobs and chooses the one with the highest wage” — and leaves it there. Written out, a firm's share of applicants is the probability that the best of nf draws lands on it:

derived, not in the paper Pi= (Fi+ si) nf − Finf

where si is firm i’s share of all posted vacancies and Fi the share posted below its wage. Setting nf = 1 collapses this to si — the wage drops out entirely.

This closed form does not appear in the paper; it is worked out here, and it is what the simulator below implements. Getting from the sentence to the formula needs two things the paper does not state: that the nf draws are uniform over vacancies rather than over firms, and that they are drawn with replacement. Both are natural readings, and with many vacancies the second makes almost no difference — but they are assumptions, not quotations.

The four rivals all post at 750, so 750 is a tie. Wages come from Eq. (1), which bounds them between 500 and 1000.

The four rivals post 5 vacancies each.

—
share of applicants preferring Firm 1
At nf = 1 the wage is irrelevant: a seeker takes whatever it drew, so applicants split in proportion to vacancies. Firm 1 holds 83% of applicants at any wage, including the floor. This is what makes the sweatshop profitable.
Raise the wage to 760 — a 1% premium — and Firm 1's share goes from 2% to 100% at nf = 20. Being marginally best is worth everything, which is what forces the bait at nf = 2 and makes bidding hopeless above it.
But this panel shows preferences, not hires. A firm can only take as many workers as it posted vacancies. When every firm keeps its posting tiny, the preferred jobs fill immediately and the surplus seekers end up in the cheap ones regardless — the rationing that makes the scarcity strategy work. It is visible in the simulator below.
Result 1 · a single RL firm

Three strategies, none of them written down

One RL firm faces four trend followers. Nothing in the reward function mentions wages, exploitation, or scarcity — only discounted profit. Yet the converged policy is qualitatively different in each of the three competition regimes, and each has a recognizable name in economics.

nf = 1 · opaque market

Sweatshop

Maximum volume, minimum wage

With no ability to compare offers, seekers must accept whatever they drew. The agent learns it can post an enormous number of jobs at the lowest possible wage and still fill them. It ends up employing most of the workforce.

Gresham's law for firms. The flood of cheap vacancies lowers everyone's fill rate, so the higher-paying trend followers face rising recruitment costs and shrink. Conscientious firms are crowded out by the exploitative one.

nf = 2 · limited competition

Bait-and-Switch Wage

Promise high, then cut

Once seekers can compare two offers, the agent must outbid to be chosen. It announces a near-maximal wage, absorbs the workforce — dragging the trend followers' wages up with it — and then, because separations do not depend on the wage, cuts pay to the floor and stops hiring.

Best regime for workers. Employment and wages are both high. Firm profits are correspondingly low, and the trend followers even run negative.

nf = 20 · near-full competition

Scarcity-Driven

Few jobs, low wages

Bidding is now hopeless: only the top wage is ever accepted, and any raise is matched. The agent instead posts as few vacancies as possible. Because its own restraint keeps the trend followers small too, the market becomes job-scarce — and scarce jobs fill even when badly paid.

Transparency backfires. The market is almost perfectly transparent, yet workers have the fewest options and a lower average wage than under limited competition.

Fig. 2

Published policies and performance

RL agent
Trend follower (averaged over the four)
Fig. 2 · Single RL firm at three levels of market transparency. Policies (vacancies, wages) and outcomes (employed workers, per-step reward) over 500 steps. Rewards are profits divided by 10⁴. Curves digitised from Fig. 2 of the published paper and redrawn as vector graphics; they reproduce the shape and scale of the originals, not their exact sample values.
Interactive

Run the market yourself

A browser reimplementation of the RL-EM environment. Choose what the RL firm does and how transparent the market is, and watch the four trend followers react over 500 steps. The point is the mechanism: why each strategy is viable in its own regime and what it does to workers.

Simulator

RL-EM, 5 firms, 500 steps

RL firm's strategy

1 = opaque · 2 = limited competition · 20 = near-full transparency

—
employed workers, market
—
average wage paid
—
RL firm · cumulative profit
—
trend follower · cumulative
step 0/500
RL firm
Trend follower (mean of four)
Market total
What is and is not the paper's system

Faithful to the paper. The wage rule (Eq. 1), the Cobb–Douglas matching function, the employment and unemployment transitions (Eq. 2–3), per-firm profit (Eq. 4), and all parameters in Table I are implemented exactly as stated.

Formalised here. The paper describes the selection rule in words — a seeker draws nf jobs and takes the highest wage — without writing down the resulting allocation. The closed form used here is the one derived in the section above, and it adds a rationing step the paper is silent on: a firm can only take as many workers as it posted vacancies, so seekers whose first choice is already full spill over to the next-best firm. Without that step the low-wage firm would receive nothing at high nf, and the scarcity strategy could not work at all.

Reimplementation choices. The trend-follower rules (Eq. 5–6) require a normalization for the wage delta that the paper does not specify; here it is divided by p−z, with a small dead band so the rule does not chatter, and vacancies floored at one. The paper does not state the firms' initial vacancy count; five is used.

Not the paper's system. The RL firm here is not a trained DDPG network. It replays a hand-specified caricature of the policy the trained agent converged to in each regime, so that the market response can be inspected step by step. Cumulative profits are therefore what a fixed strategy earns against fixed rule-based opposition, not the equilibrium payoff of a learned policy. With this setup the nf = 1 and nf = 20 market outcomes closely track the published values; at nf = 2 the trend followers do not re-expand after the RL firm withdraws, so the simulated market contracts further than the published run does.

For the published results, see the digitised figures above and below, or run the authors' code from the project repository.

Result 2 · several RL firms

Sharing a policy produces a cartel; separate policies produce factions

Replacing more of the trend followers with learners changes the market twice over. First the learners begin to set the trend rather than respond to it. Then, at NRL = 5, what matters is whether they share one policy or hold their own.

Shared actor and critic

A cartel forms on its own

With all RL firms updating the same networks, the gradient they follow is the collective one. They converge on posting many vacancies at low wages to raise joint profits — the defining behavior of a cartel — and they do so regardless of the competition level. Competition stops disciplining wages entirely.

Where competition does exist, employment and profits vary widely across the identical firms: small differences from exploration noise are amplified into very different outcomes.

Independent actors and critics

The firms segregate into strategy groups

Each firm now maximizes only its own profit, so the precondition for a cartel is gone. At nf = 1 all five converge on the sweatshop strategy. At nf = 2 they split spontaneously:

  • Three firms stop hiring from the outset, paying the lowest wage and avoiding recruitment costs.
  • One firm plays bait-and-switch: it recruits hard, then halts and cuts pay.
  • One firm keeps posting vacancies at a barely-reduced high wage, holding wages up to deter the others from re-entering the hiring market.
Summary

Final market state across all nine configurations

Employed workers (left axis)
Averaged wage (right axis)
What each configuration does to workers. Employment is nearly saturated everywhere except under near-full competition. The wage tells the real story: the highest wage in the entire study belongs to a single learner under limited competition; the shared-policy cartel at NRL = 5 suppresses wages at either competition level; independent learners restore them. Not a figure from the paper — a summary chart assembled from the market-outcome panels of its Figs. 2, 3 and 4, with values digitised from those panels. The shared-policy rows correspond to the paper's Fig. 3, which is not reproduced here.
Five RL firms with independent policies at two levels of market transparency: market summary above, per-firm vacancies, wages, employment and rewards below.
Fig. 4 · Five RL firms with independent policies. Top: market outcome at nf = 1 and nf = 2. Bottom: the policy and performance of each firm, one colour per firm. At nf = 1 the five traces coincide — every firm has found the sweatshop strategy. At nf = 2 they separate into the three groups described above. Reproduced from the paper.
Interpretation

Competition helps workers, but only up to a point

Reading the same simulations from the workers' side rather than the firms' gives the paper's policy-relevant conclusion, and it is not monotone in transparency.

No competition

Exploitation

Employment is high because the sweatshop absorbs the workforce, but wages sit at the floor and the better-paying firms have been driven out.

Limited competition · best for workers

Dynamism

Employment and wages are both at their highest across the entire study. Competition improves worker welfare without draining the market of dynamism — though firms earn very little.

Near-full competition

Manipulation

A firm able to influence market behavior responds to perfect competition by suppressing it. Job scarcity replaces wage competition, and workers end up with fewer options than before.

The broader point. None of these outcomes was encoded. Each is what profit maximization implies once the firm is allowed to discover its own policy inside a specific market structure — which is exactly what makes the combination of RL and agent-based modelling a useful instrument for policy analysis, and exactly why the market structure has to be specified with care.
Conclusion and future work

The paper introduces a framework for simulating labor markets by integrating deep multi-agent reinforcement learning into agent-based models, replacing rule-based bounded-rational firms with profit-maximizing DDPG agents. A single learner discovers three distinct strategies according to the competition level; shared-policy learners lead market trends and form a cartel; independent learners segregate into strategy groups. From the workers' perspective, outcomes are most favourable under limited competition.

AI efficiency. Multi-threading or GPU acceleration to speed up the simulations and the learning process.
Modelling extensions. RL-powered workers, enabling a game-theoretic treatment of firm–worker interaction.
Labor economics. Richer market models with heterogeneous job types or diverse worker categories.
Citation

Cite this work

@inproceedings{chen2025drl,
  author    = {Chen, Ruxin and Zhang, Zeqiang},
  title     = {Deep Reinforcement Learning in Labor Market Simulations},
  booktitle = {2025 IEEE Symposium on Computational Intelligence for
               Financial Engineering and Economics (CiFer)},
  year      = {2025},
  publisher = {IEEE},
  doi       = {10.1109/CiFer64978.2025.10975741}
}

Acknowledgement. This work was financially supported by JST SPRING, Grant Number JPiSP2125. R.C. thanks the “THERS Make New Standards Program for the Next Generation Researchers”.

An interactive presentation assembled from the paper's figures, equations and reported results. Figures 1–3 have been redrawn as vector graphics; the simulator is an independent reimplementation and is not the authors' trained system.

© 2025 IEEE. The paper has been published in 2025 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CiFer). Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. DOI: 10.1109/CiFer64978.2025.10975741.