opencode-r2

A fine-tuned object-detection model for fine-grained content detection in historical newspaper and periodical pages, built on the Beyond Words dataset (BigLam / Library of Congress). It detects the 7 page-layout content classes listed below.

Model

• Base model: facebook/detr-resnet-50 (DETR, Apache-2.0)
• Task: Object detection (COCO-style bounding boxes)
• Classes (7): Photograph, Illustration, Map, Comics/Cartoon, Editorial Cartoon, Headline, Advertisement

Dataset

• Source: biglam/loc_beyond_words
• Training: 2,846 examples
• Validation: 712 examples

Training

• Detector: DETR (facebook/detr-resnet-50) with the classification head re-initialized to 8 outputs (7 classes + no-object).
• Loss: DETR set-prediction loss (classification + L1 box + GIoU).
• Optimizer: AdamW (lr 1e-4), cosine/linear schedule with warmup, grad clip 1.0.
• Image size: shortest edge 560 px (longest edge 800 px).
• Backbone, transformer, and classifier all fine-tuned end-to-end.

Validation results

Evaluated on the 712-example biglam/loc_beyond_words validation split (COCO metrics via pycocotools).

Metric	Value
COCO mAP @ IoU [0.5:0.95]	0.2842
COCO AP @ IoU 0.5	0.4189
COCO AP @ IoU 0.75	0.3196
AR@maxDets=100	0.4302

Per-class COCO mAP @ [0.5:0.95]:

Class	mAP
Photograph	0.4027
Illustration	0.0485
Map	0.0431
Comics/Cartoon	0.2761
Editorial Cartoon	0.1702
Headline	0.5226
Advertisement	0.5259

Training loss: 21.06 (epoch 0) → 7.44 (epoch 4).

Usage

from transformers import AutoImageProcessor, AutoModelForObjectDetection
import torch
from PIL import Image

processor = AutoImageProcessor.from_pretrained("harness-race/opencode-r2")
model = AutoModelForObjectDetection.from_pretrained("harness-race/opencode-r2")

img = Image.open("page.png").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

target_sizes = torch.tensor([[img.height, img.width]])
results = processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.5)[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    box = [round(i, 1) for i in box.tolist()]
    print(f"{model.config.id2label[label.item()]}: {round(score.item(),3)} {box}")

License and attribution

• The fine-tuned weights in this repository are released under Apache-2.0 (same as the base DETR model), so the model is free to share and use.
• Base detector: End-to-End Object Detection with Transformers (Carion et al., 2020).
• Data: Beyond Words (Library of Congress) via biglam/loc_beyond_words.

Caveats

• Model was fine-tuned on a single GPU with a limited training budget; results reflect that constraint.
• Detection resolution and accuracy trade-offs exist for very small text blocks.