Meta FAIR Released Code World Model (CWM): A 32-Billion-Parameter Open-Weights LLM, to Advance Research on Code Generation with World Models

Meta fair released Code world modelA 32 billion-parameter dense decoder-keval LLM that injures World modeling In the code generation by training on execution marks and long-hurizon agents-environmental interactions-only stable source text.

What is new: Code by learning Execute execution,

Overview-CWM Middle-Training on two large families of the carrier trajectory: (1) Python interpreter mark Record local variable states after each executed line, and (2) Agent interaction inside dockerized repository This capture edits, shell command and test feedback. The purpose of this grounding is to teach semantics (how the state develops) instead of syntax.

To score the collection, the research team building Executable repository images Thousands of GITHUB projects and a software-engineering agent (“Forageragent”) inflated multi-step trajectory. Report of release ~ 3m trajectory With ~ 10k images and 3.15k across the repos, with mutant fix and issue-fix variants.

Model and reference window

CWM is one Dense, decoder-keval transformer (No mo) 64 layers, GQA (48Q/8kV), Swiglo, RmsnormAnd Plant ropeFocus alternative Local 8k And Global 131K Sliding-window block, enable 131K token Effective reference; The training uses documents-muscle masking.

Training recipe (East → Middle → Post)

• General pretense: 8T tokens in reference to 8T (code-thunder).
• Mid-train,
• Latter, 100B-token SFT + Argument for instructions, then Multi-work RL (~ 172B-token) Verification, using a GRPO-style algorithm and a minimum toolset (bash/edit/creature/submit) beyond the verification coding, mathematics and multi-turn SWE environment.
• The magnificent estimate fits on one Single 80 GB H100,

Standard

The research team cites the following Pass@1 / score (Test-time scaling was noted where applicable):

• Self-bench verified, 65.8% (With test-time scaling).
• Livecodebench-V5, 68.6%, LCB-V6, 63.5%,
• Mathematics -500, 96.6%, Aime-24, 76.0%, Aime-25, 68.2%,
• Cruxwell-output, 94.3%,

The position of the research team as CWM equally as competitive with open-weight baseline and even with large or closed models on Swe-Bench verified.

For the SWE-Bench verified work design and reference to the metrics, see the official benchmark resource.

Why does world modeling maintain the code?

The release emphasizes two operating capabilities:

1. Execution-trace prediction: Given a function and a trace start, the CWM stack frames (locals) and a structured line through a structured format at each stage predict the line -used as “nerve debun” for grounded arguments without live execution.
2. Agent coding: Multi-turn logic, hidden tests and patch equality awards with the use of equipment against actual repo; The setup trains the model to make and generate defects local End-to-end patch (GIT DIF) instead of snipet.

Some details noticeable

• Torque: Lama -3 family with reserved control tokens; Reserved ID is used to demarcate the trace and region segment during SFT.
• Meditation layout, 3: 1 local: global The interlave is repeated deeply; Long reference training occurs Big token batch size To stabilize gradients.
• Scaling calculation: Learning-vet/batch-shaped schedules are long-ranked internal scaling-lov sweeps for reference overheads.

Summary

The CWM is a practical step towards the ground generation: Meta adds a 32B dense transformer to execution-trafficking and agents, for tested patches, releases intermediate/post-practiced posts, and uses gates under a fair non-commercial research license-a useful platform for long-term fertility-a useful platform for the preparation, a useful platform for a useful stage, Makes a useful platform with execution-performance.

Check it paper, Githb pageAnd Model to hugFeel free to check us Github page for tutorials, codes and notebooksAlso, feel free to follow us Twitter And don’t forget to join us 100k+ mL subredit More membership Our newspaper,

Asif razzaq is CEO of Marktechpost Media Inc .. As a visionary entrepreneur and engineer, ASIF is committed to using the ability of artificial intelligence for social good. His most recent effort is the launch of an Artificial Intelligence Media Platform, Marktekpost, which stands for his intensive coverage of machine learning and deep learning news, technically sound and easily understand by a comprehensive audience. The stage claims more than 2 million monthly ideas, reflecting its popularity among the audience.