Recovery data for a specialist: are bird images enough?
The YOLOv8n cockpit's recovery set is 400 images that all contain birds. If the recovery data only shows positives, can the model still learn what is NOT a bird?
Methods: Recovery data for a specialist, Prune, then fine-tune (cut and regrow)
This figure did not load: the file is missing or the connection dropped. Reload the page to try again; the table below has the same numbers.
Table
Mean over the seeds; ± is the standard deviation where more than one seed ran. The experiment took 30 s.
| Data | Model | Target-class AP ('bird') | Parameters (nonzero) |
|---|---|---|---|
| everything (28k) | pruned teacher 3x32 (recovered) | 0.968 ± 0.002 | 2,792 |
| everything (28k) | fresh specialist 2x16 | 0.893 ± 0.016 | 616 |
| birds only (1403) | pruned teacher 3x32 (recovered) | 0.797 ± 0.021 | 2,792 |
| birds only (1403) | fresh specialist 2x16 | 0.721 ± 0.092 | 616 |
| birds + siblings | pruned teacher 3x32 (recovered) | 0.886 ± 0.006 | 2,792 |
| birds + siblings | fresh specialist 2x16 | 0.857 ± 0.011 | 616 |
| birds + equal # random others | pruned teacher 3x32 (recovered) | 0.962 ± 0.001 | 2,792 |
| birds + equal # random others | fresh specialist 2x16 | 0.925 ± 0.006 | 616 |
| birds + noise negatives | pruned teacher 3x32 (recovered) | 0.941 ± 0.005 | 2,792 |
| birds + noise negatives | fresh specialist 2x16 | 0.883 ± 0.013 | 616 |
| random 2806 (no bird filter) | pruned teacher 3x32 (recovered) | 0.927 ± 0.003 | 2,792 |
| random 2806 (no bird filter) | fresh specialist 2x16 | 0.873 ± 0.007 | 616 |
Reference points
- Teacher (the large model)
- The full-size network trained on the labeled data; the smaller models here are measured against it. Target-class AP ('bird') 0.989, parameters (nonzero) 140,052.
- Bayes-optimal (the best any model can do on this problem)
- Predictions from the exact class probabilities, which the lab can compute because it generated the data. Target-class AP ('bird') 0.999.
Birds in this experiment
The table's 12 smaller models hatch 2 kinds of bird. Each kind is drawn once, for one model: the smallest if the bird is healthy or bones, the one that keeps the most if it's starving.
- Ruby-throated Hummingbird, healthyData: everything (28k); model: pruned teacher 3x32 (recovered)50.1× smaller, keeps 97.9% of the teacher's target-class APBest of 3 models
- Ruby-throated Hummingbird, starvingData: random 2806 (no bird filter); model: pruned teacher 3x32 (recovered)50.1× smaller, keeps 93.8% of the teacher's target-class APBest of 9 models
Measured against this experiment's teacher: 140,052 parameters, 4481.7 kbit, target-class AP 0.989. Storage is 32 bits a parameter, the lab's rule for a dense fp32 net, which every model drawn here is. Species follows how many times smaller a model is. A bird stays healthy while its model keeps at least 95% of the reference's score, and turns to bones at 60% or less.
These are the lab's recorded results, so they aren't added to anyone's life list. Birds you hatch yourself, in the Coop or from a Field Guide run, are saved to your life list.