---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: image-classification
inference: false
base_model: timm/vit_base_patch16_224.augreg_in21k_ft_in1k
tags:
  - facial-expression-recognition
  - driver-monitoring
  - vision-transformer
  - parameter-efficient-fine-tuning
  - lora
  - adaptformer
  - timm
metrics:
  - accuracy
  - f1
---

# Parameter-Efficient Adaptation of Facial Expression Recognition Models for Driver Monitoring

Weights and evaluation records for **Ayush Tiwari's MSc thesis**, BTU
Cottbus-Senftenberg, 2026. The four strategies reported in the thesis are
**full fine-tuning, linear probe, AdaptFormer and LoRA**.

An ImageNet-pretrained ViT-B/16 is fine-tuned on six-class FER+ (Stage 1),
then adapted to near-infrared KMU-FED face crops with a new classifier (Stage 2).
The standalone loader below includes the model definitions and preprocessing.

## Results

KMU-FED: 1,106 frames from 12 drivers, ten frame-block folds, 110 held-out
entries per fold. Accuracy is the ten-fold mean ± sample standard deviation;
weighted F1 pools all 1,100 held-out entries, matching the thesis.

| Strategy | Selected setting | Trainable parameters | Accuracy (%) | Pooled weighted F1 (%) | Errors / 1,100 |
|---|---|---:|---:|---:|---:|
| Full fine-tuning | lr 3e-5 | 85,803,270 | 97.55 ± 4.11 | 97.54 | 27 |
| Linear probe | lr 3e-3 | 4,614 | 93.27 ± 6.83 | 93.22 | 74 |
| AdaptFormer | lr 1e-3, width 64 | 1,194,246 | 98.45 ± 2.43 | 98.45 | 17 |
| LoRA | lr 1e-3, rank 4 | 152,070 | 99.00 ± 2.59 | 99.00 | 11 |

FER+ six-class official test: **93.21% accuracy**, **93.13% weighted F1**
over 2,047 retained images. Learning rate 1e-5 and epoch 28 were selected on
the official validation split. This study's projection excludes images with
neutral/contempt at the maximum vote, including ties; these scores are not the standard
eight-class FER+ benchmark.

Both epoch and configuration selection use the reported KMU-FED folds, so the
scores are optimistic, best-observed results. Drivers and sequences recur across
folds; six byte-identical alias pairs also cross training and evaluation.
FER+ has identical images across its official splits. The comparison is
descriptive, with one base seed (42); it does not establish unseen-driver
generalisation, statistical equivalence, inference speed or the benefit of
Stage 1. There is no all-data deployment model.

## Use a checkpoint

Use **Python 3.11** and the bundled loader for these custom PyTorch checkpoints.
From a fresh working directory, download only the inference support files:

```bash
python -m pip install "huggingface_hub>=0.25,<1"
python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(
    "headless-start/parameter-efficient-dfer",
    allow_patterns=["load_weights.py", "requirements.txt"],
    local_dir="dfer_weights",
    token=False,
)
PY
python -m pip install -r dfer_weights/requirements.txt
```

```python
import sys
import torch
from PIL import Image

sys.path.insert(0, "dfer_weights")
from load_weights import CLASS_NAMES, build_transform, load_stage2

model = load_stage2("lora", fold=0)  # CPU; device="cuda" for a GPU
with Image.open("face_crop.jpg") as image:
    x = build_transform()(image.convert("RGB")).unsqueeze(0)
with torch.inference_mode():
    probabilities = model(x).softmax(dim=1)[0]
print(CLASS_NAMES[probabilities.argmax().item()])
```

`load_stage1()` loads the FER+ model. `load_stage2(method, fold)` accepts
`full_ft`, `linear_probe`, `adaptformer` or `lora`, with zero-based folds
`0`–`9`. Both accept `device` and `revision` keyword arguments; pin a Hub
commit with `revision="<commit-sha>"` for repeatable downloads. Models return
in evaluation mode; move input tensors to the same device as the model.

Class order: **angry, disgust, fear, happy, sad, surprise**. Input must already
be a face crop. The transform produces three-channel grayscale, bicubic
224 × 224 resizing and mean/std 0.5 normalisation. Softmax values are
uncalibrated model scores. The command-line equivalent is:

```bash
python dfer_weights/load_weights.py face_crop.jpg --method lora --fold 0
```

## Files and model details

**41 checkpoints**: one FER+ model and ten folds for each of the four KMU-FED
strategies. Only selected configurations are downloadable; their candidate
scores remain in the search summaries. No dataset images are distributed.

| Path | Contents |
|---|---|
| [`stage1/`](https://huggingface.co/headless-start/parameter-efficient-dfer/tree/main/stage1) | Complete model (~343 MB), test results and learning-rate study |
| [`stage2/`](https://huggingface.co/headless-start/parameter-efficient-dfer/tree/main/stage2)`<method>/fold_00/` … `fold_09/` | Weights, `results.json` (including SHA-256) and per-image `predictions.json` |
| `stage2/<method>/selection.json`, `search_summary.json` | Selected settings and candidate scores |
| [`load_weights.py`](https://huggingface.co/headless-start/parameter-efficient-dfer/blob/main/load_weights.py), [`requirements.txt`](https://huggingface.co/headless-start/parameter-efficient-dfer/blob/main/requirements.txt) | Standalone inference and dependencies |

Full fine-tuning stores complete models (~343 MB each). Other folds store
only trained parameters and the head: linear probe ~22 KB, AdaptFormer ~4.8 MB,
LoRA ~628 KB. Their loader also downloads the Stage-1 model once and reuses it
from the local cache. AdaptFormer uses a parallel MLP adapter, width 64,
scale 0.1 and dropout 0.1; LoRA adapts Q/V in all 12 blocks with rank 4 and
alpha/r = 2. Both retain the six-class head. All Stage-2 strategies use AdamW,
batch size 32 and a 60-epoch budget, with two warm-up epochs and cosine decay.
The JSON records preserve the original experiment schema and numerical results.

## Licence and citation

Weights: **CC BY-NC 4.0**. Dataset and pretrained-model terms remain with
their providers. Intended for research on facial-expression adaptation;
operational driver monitoring was not validated.

```bibtex
@mastersthesis{tiwari2026peft_dfer,
  author = {Ayush Tiwari},
  title = {Parameter-Efficient Adaptation of Facial Expression Recognition Models for Driver Monitoring},
  school = {Brandenburgische Technische Universit{\"a}t Cottbus-Senftenberg},
  year = {2026}
}
```