--- license: mit task_categories: - text-generation tags: - code - reinforcement-learning - rlvr configs: - config_name: default data_files: - split: train path: train.parquet - split: test path: eval.parquet --- # AceCode-87K train / held-out test split (verl format) The exact train and test data used for the binary-reward code RL runs on Qwen3-1.7B-Base (GRPO / MaxRL) in [max-rl](https://huggingface.co/max-rl). Source: [TIGER-Lab/AceCode-87K](https://huggingface.co/datasets/TIGER-Lab/AceCode-87K) (AceCoder, arXiv:2502.01718), which ships a single `train` split: function-level Python problems, each with GPT-4o-mini-synthesized `assert` test cases. | file | split | rows | `data_source` | |---|---|---|---| | `train.parquet` | train | 85,847 | `acecode` | | `eval.parquet` | test (held-out) | 1,000 | `acecode_eval` | Train and test share no AceCode `id` (overlap 0). Per-split counts are in `stats.json`. ## How the split was made `preprocess_acecode_data.py` (included) on the full 87,149 rows: 1. shuffle with seed 42; 2. keep only `assert ...` test strings; drop rows with < 3 tests, empty questions, or a duplicate question (302 rows dropped); keep at most 16 tests per prompt; 3. drop prompts longer than 3,072 tokens under the Qwen3-1.7B-Base chat template (0 dropped); 4. the **last 1,000** surviving rows are the test split, the rest are train. ## Schema (verl RL parquet) - `prompt`: chat messages, one user turn = the AceCode question + the suffix `"\nPlease reason step by step, then write a self-contained Python solution. Give your final program in a single ```python code block."` - `ability`: `"code"` - `reward_model`: `{"style": "rule", "ground_truth": "{\"tests\": [\"assert ...\", ...]}"}` - `extra_info`: `split`, `index`, `acecode_id`, `num_tests` ## Reward Strictly binary: take the last ```python block of the response, append all `assert` tests, run in a fresh `python -I` subprocess (10 s wall / CPU limit, 4 GiB address space). Score 1.0 iff the process exits 0, else 0.0 (no code block → 0.0). Validated 60/60 against AceCode-87K's own `pass_rate ∈ {0, 1}` inference labels. Evaluation in the runs: 4 samples per test prompt (T=0.6, top-p 0.95), reporting mean@4 and best@4.