Examples & Tutorial Scripts¶
A curated catalog of runnable astronomical examples, from survey mosaic processing and IFU datacubes to X-ray event table filtering and neural network training.
All example scripts live in the examples/ directory of the repository, and are mirrored in published-examples/.
To run the automated validation suite across all example scripts:
python examples/test_examples.py
Sample Data Sources¶
Most examples run against public astronomical survey and observatory data. Sample data can be downloaded into your local cache using the repository helper scripts:
# Fetch Astropy tutorial samples (HorseHead, M13, Chandra, SDSS)
bash scripts/fetch_example_samples.sh
# Fetch SDSS MaNGA DR17 LOGCUBE (~200 MB)
bash scripts/fetch_example_samples.sh --with-manga
# Fetch CFHT MegaCam 36-CCD MEF exposures (CADC)
bash scripts/fetch_cfht_megacam_sample.sh
# Fetch CFHT MegaPipe survey mosaic stacks (~5.3 GB)
bash scripts/fetch_cfht_megapipe_sample.sh
1. Survey Mosaics & Multi-Extension FITS (MEF)¶
Astronomical cameras (such as CFHT MegaCam, Subaru Hyper Suprime-Cam, and DECam) produce Multi-Extension FITS files comprising dozens of individual CCDs.
Extracting Cutouts from Multi-CCD Exposures¶
Extract postage stamps across individual CCD extensions (EXTNAME="CCD01", CCD02, ...) using high-throughput subset readers:
import torchfits
# Reusable reader handle for fast repeated extractions on a 36-CCD exposure
with torchfits.open_subset_reader("megacam_exposure.fits.fz", hdu="CCD01") as reader:
stamp1 = reader.read_subset(100, 100, 228, 228)
stamp2 = reader.read_subset(500, 500, 628, 628)
- Script:
example_megacam_mef_cutouts.py - Gigapixel Collage Demo:
example_megapipe_cutout_collage.py
Cutouts with World Coordinate System (WCS) Updates¶
When extracting a sub-region from an image, the reference pixel coordinates (CRPIX1, CRPIX2) in the FITS header must be translated to preserve astrometric calibration. torchfits does not own a WCS engine: the torchfits cutout command shifts CRPIX* by the cutout offset for you, but in Python you translate them yourself:
import torchfits
# Extract cutout and compute translated WCS reference pixel coordinates
cutout, header = torchfits.read("spitzer_irac.fits", return_header=True)
header["CRPIX1"] -= 100
header["CRPIX2"] -= 100
torchfits.write(
"spitzer_cutout.fits", cutout[100:356, 100:356], header=header, overwrite=True
)
- Script:
example_cutout_wcs_write.py
2. Spectroscopy & 3D Datacubes¶
SDSS MaNGA IFU Datacubes¶
SDSS-IV MaNGA (Mapping Nearby Galaxies at APO) packages 3D integral field unit (IFU) spectroscopy into multi-extension FITS datacubes containing flux, inverse variance, mask planes, and wavelength grids:
import torchfits
# Read 3D spectral flux cube [Wavelength, Y, X]
flux_cube = torchfits.read_tensor("manga_logcube.fits", hdu="FLUX")
ivar_cube = torchfits.read_tensor("manga_logcube.fits", hdu="IVAR")
wave_grid = torchfits.read_tensor("manga_logcube.fits", hdu="WAVE")
print(f"Datacube shape: {flux_cube.shape}") # [4563, 74, 74]
- Script:
example_manga_logcube.py - Radio Cube Script:
example_cube.py
3. Astronomical Event Tables & Catalogs¶
Chandra X-Ray Event Lists¶
High-energy astrophysics instruments record individual photon arrival events. Query and filter multi-million photon event files using C++ pushdown predicates directly into PyArrow:
import torchfits
# Filter Chandra X-ray events by energy band (e.g. hard X-ray band > 5 keV)
events = torchfits.table.read(
"chandra_events.fits",
hdu="EVENTS",
columns=["time", "x", "y", "energy"],
where="energy > 5000",
)
print(f"Selected {events.num_rows} hard X-ray photons.")
- Script:
example_table.py - DuckDB & Polars Analytics:
example_table_recipes.py
4. Multi-Exposure Coaddition & Image Stacking¶
Combine dithered telescope exposures into a high signal-to-noise coadded image tensor:
import glob
import torch
import torchfits
# Stack multiple raw exposures (e.g. Messier 13 globular cluster frames)
frames = [torchfits.read_tensor(f, hdu=0).float() for f in glob.glob("m13_blue_*.fits")]
stacked_image = torch.stack(frames, dim=0).mean(dim=0)
torchfits.write("m13_coadd.fits", stacked_image, overwrite=True)
- Script:
example_m13_stack.py
5. Storage Optimization: Robust 16-Bit Quantization¶
When storage budgets or bandwidth require compressing 32-bit floating-point astronomical images and table columns into 16-bit integers (BITPIX=16, TFORM=I), standard global min-max scaling loses dynamic range to cosmic rays and noise spikes. The quantize="robust" parameter automatically uses percentile-based bulk packing:
import torchfits
image_float = torchfits.read_tensor("raw_float.fits")
# Pack into 16-bit integer FITS with automatically calibrated BSCALE and BZERO
torchfits.write("packed_int16.fits", image_float, quantize="robust", overwrite=True)
- Script:
example_quantize_int16.py
6. Complete Example Scripts Directory¶
Image & Datacube Processing¶
| Script | Purpose & Key APIs |
|---|---|
example_image.py |
Basic read and write round-trip for 2D images |
example_image_cutouts.py |
Bounding box cutouts with read_subset and open_subset_reader |
example_cube.py |
3D radio and spectral data cube slicing |
example_image_mef.py |
Multi-extension inspection and named HDU extraction |
example_m13_stack.py |
Multi-frame exposure stacking and median coaddition |
example_mef_header.py |
Navigating complex headers across multi-CCD files |
example_cutout_wcs_write.py |
Extracting sub-regions and updating CRPIX astrometry keywords |
example_manga_logcube.py |
SDSS MaNGA IFU datacubes (FLUX, IVAR, MASK, WAVE) |
example_megacam_mef_cutouts.py |
Parallel cutouts from CFHT MegaCam 36-CCD mosaics |
example_quantize_int16.py |
Robust 16-bit quantization for images and table columns |
Tables & Catalogs¶
| Script | Purpose & Key APIs |
|---|---|
example_table.py |
Arrow tables, tensor dictionary loading, and Chandra event filtering |
example_polars.py |
read_polars and out-of-core scan_polars streaming |
example_table_recipes.py |
SQL pushdown, DuckDB querying, and Arrow record batch iterators |
Datasets & Streaming¶
| Script | Purpose & Key APIs |
|---|---|
example_staged_cutouts.py |
FitsStagedCutoutIterableDataset: background prefetch of the next survey mosaic, bounded scratch staging, DataLoader integration |
example_streaming_cubes_spectra.py |
FitsCubeIterableDataset and FitsSpectrumIterableDataset for constant-memory IFU cubes and multi-arm spectra |
Correctness Cookbooks & Harnesses¶
These are assertion-driven rather than illustrative: every check below raises on failure, so the smoke suite reports a broken read or write instead of printing it.
The fitsio and CCfits cookbooks live in torchfits-harness.
| Script | What it asserts |
|---|---|
example_identity_stress.py |
The same pixels and columns through every write/read path: write/write_tensor/HDUList, tensor dicts, scan batch sizes, where= row counts, uint16, VLA + 2-d columns, RICE_1 and GZIP_1 |
Machine Learning & Preprocessing¶
| Script | Purpose & Key APIs |
|---|---|
example_ml_galaxyzoo_legacy.py |
Galaxy Zoo 1 morphology classification on Legacy Survey cutouts |
example_ml_training_loop.py |
Raw FITS to a training step: band discovery, DQ masks, data-state contract, spectra/IFU/catalog loaders |
example_megacam_cr_denoise.py |
FITS-native Noise2Noise calibration, held-out dark test, and conservative science CR repair |
example_megapipe_cutout_collage.py |
Gigapixel mosaic cutout extraction and Lupton RGB collage |
example_lupton_rgb_sdss.py |
Lupton asinh RGB synthesis from SDSS \(g, r, i\) filter frames |
example_rgb_sky.py |
Auto RGB collage: Legacy Survey Virgo, JWST SMACS 0723, HST OPAL Jupiter |
example_transforms.py |
Composite stretch and normalization pipelines (Compose) |
example_time_series.py |
Light curve filtering with symmetric and asymmetric sigma clipping |
example_custom_transform.py |
Implementing custom FITSTransform subclasses |
example_make_loader_vs_dataloader.py |
Benchmark comparing make_loader cache warmup vs DataLoader |
example_image_dataset.py |
Minimal FitsImageDataset + make_loader (ML guide) |
example_data_catalogs.py |
Table + cutout datasets (ML guide) |
Figure generators¶
| Script | Output |
|---|---|
gallery_images.py |
image before/after PNGs |
gallery_tables_lc.py |
light-curve / table plots |
Samples use TORCHFITS_SAMPLE_CACHE when set, otherwise torchfits' normal
cache precedence. examples/test_examples.py sets TORCHFITS_EXAMPLE_FAST=1
for every example under CI, and unconditionally for
example_megacam_cr_denoise.py, to skip downloads and bound its training run.
The runner also refuses to start under PYTHONOPTIMIZE, which would delete the
assertions three of these examples are built from.
Out of gallery¶
These are published alongside the gallery but are not themselves part of the image/table figure pipeline:
desi_shaped_spectrum.py— DESI-shaped multi-arm spectrum demo (API data)cli/make_rgb_demo.py— synthetic RGB collage generator for CLI recipes (CLI recipes)cli/imstat_imarith.sh—imstat/imarith-style walk through ninetorchfitsCLI subcommands on the HorseHead sampletest_examples.py— the smoke runner itself: it executes every other example on this page and is the gatedocs-contractruns