Procedural Pile

Verifiable reasoning problems, generated by code, formatted as supervised training data.

Procedural Pile contains 24,806,238 problems from 55 task families: arithmetic, equation systems and inverse word problems, first-order logic and formal proofs in Metamath, program execution and state tracking, belief tracking, planning, graph search, grammars, attribute grammars and regular expressions, SQL over tables, games, causal and probabilistic inference, and more. Every answer is computed by a solver or checked by a verifier. None are written by a language model.

Most procedural reasoning collections are built as RL environments. Procedural Pile is built as training data: for pretraining, mid-training, and supervised fine-tuning. Every task and difficulty range was kept because its measured effect as SFT data justified it. In the Reasoning Core paper, a 3B model trained on its first release reaches the highest mean scores on DROP, LogiQA, and ARC-Challenge. It beats a matched baseline without procedural data, and it beats three other procedural collections: Reasoning Gym, SynLogic, and Procedural Warmup.

Designed for SFT, and measured as SFT

• Measured SFT transfer. Each task is fine-tuned in isolation, and its effect on held-out reasoning and on general text likelihood is measured. These measurements decide which tasks are kept, how difficult they are, and how their answers are written. They are not proxies such as solver accuracy or diversity scores.
• Broad structure, not surface variation. Generators sample the underlying structure of each problem: expression trees, logical forms, grammars, and program structures. Planning and game tasks draw new rule systems, not just new instances of Blocksworld or tic-tac-toe.
• Compact, canonical targets. Answers are short: a number, a label, a set, a plan, a proof step, a short program. When several answers are valid, one canonical answer is the training target. The token budget goes to more distinct problems, not to explanations. In the paper, step-by-step solver traces trained worse than the answers they compute.
• Calibrated difficulty. Each task has a continuous difficulty knob mapped to its own parameters, such as derivation depth, plan length, or branching factor. Levels are tuned so that tasks stay learnable.
• Audited. Generators, prompt rendering, targets, and scorers are tested separately and reviewed at repository scale, so ambiguous or mis-scored instances are caught before release.

Quick start

from datasets import load_dataset

ds = load_dataset("reasoning-core/procedural-pile", split="train", streaming=True)
print(next(iter(ds))["prompt"])

The dataset has train and test splits and needs no configuration name. The default revision is the latest release, rc14. Every release is tagged, so an earlier one stays loadable, for instance revision="rc13". To reproduce the paper, pin its snapshot with revision="paper-2608.05148".

Example

[controlled_code_execution]

Predict the value returned by this Python call.

def endpoint():
    state = [2, -2, -3]
    alias0 = state
    alias0[2] += -3
    state[1] += alias0[2]
    bias1 = 4
    def f1(x, bias=bias1):
        return x + bias + state[1]
    bias1 += -3
    state[2] = f1(state[2])
    return state

Call: endpoint()
The answer is the exact Python repr of the returned value.

→ [2, -8, -10]

The task gallery shows a worked example for every task.

Fields

Tasks

• Logic & deduction · 12: logic_qa, multistep_nli, multistep_abduction, multistep_evidence_retrieval, defeasible_nli, logic_derivation, unification_entailment, rewrite_system, lambda_reduction, metamath_core_select, metamath_entailment, analogical_case_matching
• State tracking & execution · 11: process_inversion, systems_trace, belief_tracking, reference_tracking, rule_switching, finite_automaton_execution, shift_reduce_parsing, controlled_code_execution, code_runnability, code_analysis, dynamic_programming
• Mathematics · 7: math_word_problem, inverse_math, combinatorics_formula, equation_system, arithmetics, function_manipulation, sequential_induction
• Spatial, causal & probabilistic · 7: planar_geometry_relations, constraint_satisfaction, qualitative_causal_reasoning, qualitative_reasoning, grid_navigation, most_probable_outcome, most_probable_evidence
• Formal languages & parsing · 7: attribute_grammar, regex_reasoning, regex_following, constrained_continuation, syntax_error_detection, parsing_derivation, string_transduction
• Graphs, games & planning · 6: game_forced_win, game_best_move, graph_pathfinding, graph_successors, planning, coreference
• Sets & tables · 5: set_expression, set_missing_element, table_qa, table_equivalence, table_statistics

Releases

• rc14 (current). 55 tasks.
  • New: belief_tracking, process_inversion, systems_trace, attribute_grammar, inverse_math.
  • Removed: program_synthesis.
  • Changed: multistep_evidence_retrieval lists its premises in one canonical order, so each problem has a single canonical answer.
• rc13. 51 tasks, 25,457,455 rows.
• paper-2608.05148. The snapshot used in the paper.

Beyond SFT

Each row carries what its task's verifier needs, so the same data can serve as a reward source for RL with reasoning_core.score_answer(completion, row). Fresh problems at any difficulty come from the Reasoning Core library (pip install "reasoning-core[gen]"; plain reasoning-core is enough to score).

Citation

@article{reasoningcore2026,
  title={Reasoning Core: Designing Broad Procedural Data for Completion-Supervised Reasoning Training},
  author={Sileo, Damien and Lacombe, Valentin and Kachler, Dimitri},
  journal={arXiv preprint arXiv:2608.05148},
  year={2026}
}