CHRIS-PROBA1 v2: cloud and cloud-shadow segmentation

Segmentation of clouds and cloud shadows in CHRIS/PROBA-1 imagery, into three classes: clear, cloud and shadow. It takes four bands (R, G, B, NIR) built from the CHRIS acquisition mode and returns a label map for the whole scene.

pip install mlstac pytorch-lightning segmentation-models-pytorch timm rasterio
import mlstac

model = mlstac.load("https://huggingface.co/isp-uv-es/CHRIS-PROBA1/resolve/main/mlm.json")
local = model.download("CHRIS-PROBA1")
net = local.compiled_model(device="cuda")

mask = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3, source="toa")
# 0 = clear, 1 = cloud, 2 = shadow, 255 = nodata

metrics

Accuracy

On 55 hand-painted CHRIS scenes, split by flight and never seen in training, each scene going through the full inference path in load.py:

Metric Value Aggregation
mIoU 0.708 mean per scene
IoU clear / cloud / shadow 0.898 / 0.831 / 0.652 pixels pooled
Macro F1 0.881 pixels pooled
Cohen's kappa 0.854 pixels pooled
Accuracy 0.921 pixels pooled

The first row averages scenes; the rest pool every valid pixel. They are not interchangeable: the pooled mIoU is 0.793, the per-scene mean 0.708.

Per mode, mean mIoU and number of scenes: mode 1 0.784 (12), mode 2 0.697 (5), mode 3 0.699 (12), mode 4 0.576 (5), mode 5 0.658 (3), mode 6 0.711 (18). Modes 2, 4 and 5 have five scenes or fewer, so those numbers are indicative only.

How it works

Three U-Nets, averaged:

Checkpoint Encoder
checkpoints/convnextv2_a.ckpt ConvNeXtV2-nano
checkpoints/convnextv2_b.ckpt ConvNeXtV2-nano
checkpoints/regnety.ckpt RegNetY-004

The two ConvNeXt share architecture and bands; they differ in the labels they learned from, so they miss different pixels and the average is steadier than any one of them.

Inference walks the scene in 640 px windows with 320 px overlap, snaps the last window to the edge so nothing is padded, normalises each window per band with a z-score, averages the softmax of the three checkpoints and blends the windows with a gradient mask. Around 0.23 s per 748 x 744 scene on an RTX A5000 with the weights loaded.

example

Inputs

The four bands are the ones that best reproduce Sentinel-2 B4, B3, B2 and B8 for each acquisition mode, from the cosine similarity between the CHRIS spectral response and the S2 bands (bands.json).

File source Notes
TOA product "toa" reflectance x10000, the usual case
Radiance "rad" needed for mode 20, which has no TOA product
Raw DN "dn" accepted, less accurate

Modes 1 to 5 and mode 20 (passed as mode_n=6) are supported. Mode 1 is binned to 372 x 374 by the instrument, so its window shrinks to 352 px; the others are 744 px wide, with 748 rows in most acquisitions and up to 1024 in some mode 2 products.

With your own four-band stack:

mask = local.module.predict_large(rgbn, model=net)   # rgbn: (4, H, W) float

Per-pixel confidence:

probs = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3,
                                   source="toa", return_probs=True)

Walkthrough with the six modes and a batch example: CHRIS-PROBA1-demo.ipynb.

modes

all modes, input and mask

Data

  • Pretraining: CloudSEN12, 50,247 patches resampled to 640 x 640 at 10 m with 13 spectral bands, labelled clear / cloud / shadow. Eleven band combinations were trained with ConvNeXtV2-nano and RegNetY-004. Those with four bands or more land within 0.008 mIoU of each other in our runs, so the four-band one was kept. It is the cheapest and the only one every CHRIS mode can build.
  • Fine-tuning: 130 hand-painted CHRIS scenes, each sampled 15 times with random crops, plus 1,611 automatic masks over scenes nobody painted. A second round retrained on the model's own predictions, keeping a pixel only where the prediction was confident and stable.
  • Test: 55 painted scenes, split by flight, never by scene, so two angles of the same acquisition never sit on both sides.

What it is not

  • Shadow is the weak class (IoU 0.652). CHRIS covers 406 to 1005 nm: no SWIR, no thermal band, so a shadow and a dark surface look alike. Expect misses over water and dense vegetation. A shadow mask driven by sun geometry and a DEM beats this model on those pixels.
  • No thin-cloud class. Thick and thin are merged into cloud.
  • Differences under 0.03 mIoU are noise with 55 test scenes.
  • Mode 2 after 2015 is unusable: those products store a gain setting of -1, so the raw calibrated imagery (RCI) is not calibrated and the digital numbers are meaningless.
  • The three checkpoint files carry the optimizer state, which is why they weigh more than the networks.

Files

load.py                 mlstac loader
mlm.json                model card, STAC/mlm format
bands.json              CHRIS to RGBN table per acquisition mode
checkpoints/            the three networks
examples/mode_<n>/      one scene per mode, with its painted mask and the prediction
figures/                metrics, example, gallery
CHRIS-PROBA1-demo.ipynb
metrics.json            per-class and per-mode numbers

Citation

@misc{chrisproba1_v2,
  title  = {CHRIS-PROBA1: cloud and cloud-shadow segmentation for CHRIS/PROBA-1},
  author = {Contreras, Julio and {ISP-UV}},
  year   = {2026},
  howpublished = {https://huggingface.co/isp-uv-es/CHRIS-PROBA1}
}

MIT licence.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support