Skip to content

Core I/O Reference

Fundamental read and write operations for FITS images, tables, headers, multi-extension files, checksums, and caching.


Missing files and HDUs

Every route that takes a path and an hdu (the readers on this page, open()[...], the skinny metadata calls, checksums, HDU mutation, the table readers and mutators, and the datasets) fails the same way:

Failure Exception
Local path does not exist FileNotFoundError (an OSError, and also a RuntimeError)
EXTNAME / HDUNAME that no HDU carries torchfits.HDUNotFoundError
Integer hdu that is negative or past the last HDU torchfits.HDUNotFoundError

HDUNotFoundError subclasses KeyError, IndexError, ValueError and RuntimeError, so code that caught any of those keeps working. The missing-file error is a FileNotFoundError subclass that is also a RuntimeError, the type most readers raised for it in 1.3.1, so an except RuntimeError written against 1.3.1 still catches both. Its message names the request and the file, and the attributes hdu, path and n_hdus carry the same values (n_hdus is None when the count is unknown):

import torchfits

try:
    sci = torchfits.read("mef.fits", hdu="SCI")
except torchfits.HDUNotFoundError as exc:
    print(exc)  # HDU 'SCI' not found in mef.fits (3 HDUs)
except FileNotFoundError as exc:
    print(exc)  # FITS file not found: mef.fits

An hdu of the wrong type (a float, or None where autodetection is not offered) is still a TypeError or ValueError. HDUList indexing keeps Python sequence rules: hdul[-1] is the last HDU, and only an index outside -len(hdul)..len(hdul)-1 raises HDUNotFoundError. Remote URLs keep the errors of their transport.


read()

Unified FITS reader. Auto-detects image or table HDUs.

torchfits.read(
    path,
    hdu=0,
    device="cpu",
    mmap="auto",
    mode="auto",
    return_header=False,
    *,
    fp16=False,
    bf16=False,
    raw_scale=False,
    scale_on_device=True,
    use_cache=None,
    columns=None,
    start_row=1,
    num_rows=-1,
    cache_capacity=10,
    fast_header=True,
)
Parameter Type Default Description
path str, PathLike, or sequence (required) One FITS path or a list/tuple of paths; a sequence returns one result per path
hdu int, str, sequence, or None 0 HDU index, EXTNAME, a list/tuple of HDUs from one path, or None/"auto" for autodetection
device str "cpu" "cpu", "cuda", "mps"
mmap bool or str "auto" True, False, or "auto"
mode str "auto" "auto", "image", or "table"
return_header bool False Return (data, Header) tuple
fp16 bool False Convert image results to torch.float16; if both precision flags are true, fp16 takes precedence
bf16 bool False Convert image results to torch.bfloat16
raw_scale bool False Return image storage values without applying BSCALE/BZERO
scale_on_device bool True For non-CPU image reads, transfer logical values directly; False selects the host scaling path before transfer
use_cache bool or None None False disables the read cache; True enables it (using capacity 10 when cache_capacity <= 0); None follows cache_capacity
columns list[str] or None None Table columns to return; invalid with mode="image"
start_row int 1 1-based first table row; invalid with mode="image"
num_rows int -1 Number of table rows (-1 reads through the end; 0 returns every column empty, with its usual dtype); invalid with mode="image"
cache_capacity int 10 Read-cache capacity; 0 disables cached reads
fast_header bool True Use the fast header path in the unified-reader fallback

Returns: torch.Tensor (images), a column mapping for tables (scalar columns are tensors; VLA columns may use list/tuple values), or a tuple if return_header=True. Multiple paths or HDUs return results in input order.

The options after return_header are keyword-only. Unknown keywords raise TypeError; a use_cache that is not a boolean or None raises ValueError.

When to mmap

Mmap can help eligible large local integer IMAGE HDUs and repeated cutouts. It still returns a normal tensor, and float / compressed images use a buffered path. Prefer mmap=False when many workers open the same files, on cold network filesystems, and for VLA / scaled tables. Dataset docs: Data module.

When to use

Use read() for quick exploration (hdu=0 by default; hdu=None autodetection). Prefer read_tensor() for images and table.read() for catalogs with where= / Arrow. Root read() on a table HDU returns a column → tensor dict and has no where= parameter.

# Image with header
data, hdr = torchfits.read("image.fits", hdu=0, return_header=True)

# Auto-detect table → column → tensor dict
columns = torchfits.read("catalog.fits", hdu=1)

read_tensor()

Read any N-dimensional FITS array directly as a PyTorch Tensor.

torchfits.read_tensor(
    path,
    hdu=0,
    device="cpu",
    mmap=True,
    fp16=False,
    bf16=False,
    raw_scale=False,
    return_header=False,
    fallback_get_header=None,
)
Parameter Type Default Description
path str (required) FITS file path
hdu int or str 0 HDU index or EXTNAME
device str "cpu" "cpu", "cuda", "mps"
mmap bool True Request the eligible memory-mapped input path
fp16 bool False Read as float16
bf16 bool False Read as bfloat16
raw_scale bool False Skip BSCALE/BZERO, return native storage dtype
return_header bool False Return (tensor, Header)
fallback_get_header Callable or None None Internal: replaces the reader used to build the returned Header objects. Present so read_hdus() can share one header reader; not for application code

Returns: torch.Tensor (or tuple if return_header=True).

When to use

Read an IMAGE HDU as a single tensor (spectrum, image, or cube). For multi-extension files use read_hdus(); for cutouts use read_subset().

GPU reads

Pass device="cuda" or device="mps" to place the result on device. Generic BSCALE/BZERO scaling yields float32 for BITPIX 8/16 and float64 for BITPIX 32/64 unless you opt into storage dtypes with raw_scale=True. For example, signed-byte and pseudo-unsigned conventions expose their on-disk uint8 / int16 storage dtypes rather than the logical int8 / uint16 dtypes.

# Read to GPU
tensor = torchfits.read_tensor("image.fits", hdu=0, device="cuda")

# Native integer dtype (matches fitsio)
tensor = torchfits.read_tensor("image.fits", hdu=0, raw_scale=True)

# 3D cube
cube = torchfits.read_tensor("cube.fits", hdu=0)
# cube.shape = (nz, ny, nx)

read_subset()

Read a rectangular pixel subset from an image HDU. On 3D+ cubes, the window applies to the trailing (y, x) axes; the leading depth axis is kept in full.

torchfits.read_subset(path, hdu, x1, y1, x2, y2)
Parameter Type Default Description
path str (required) FITS file path or HTTP(S) URL
hdu int or str (required) HDU index or EXTNAME
x1, y1, x2, y2 int (required) Half-open pixel window [x1,x2)×[y1,y2)

Returns: torch.Tensor

HTTP(S) uncompressed 2D images Range-fetch a row-band (no full download). Compressed / scaled remotes and vos: / vault: paths materialize into the remote cache first, then cut out locally.

stamp = torchfits.read_subset("mosaic.fits", hdu=0, x1=0, y1=0, x2=256, y2=256)

CFITSIO image sections on the path (1-based inclusive) work the same way users expect from imcopy / CFITSIO, e.g. read_tensor("mosaic.fits[1:256,1:256]") or torchfits cutout 'mosaic.fits[1:256,1:256]' out.fits. For Python / NumPy-style windows use read_subset or CLI --box (0-based half-open). Do not stack a path section with read_subset / --box on the same call. Binspec / complex CFITSIO filters are not a certified torchfits surface — use table.read(..., where=) for catalog predicates.

A section is recognised only when it ends the file name. A [ elsewhere, for example in a directory such as /data/[run1]/img.fits, is part of the path and every read and write works there; a section on such a path (/data/[run1]/img.fits[1:10,1:10]) is not supported, because CFITSIO would read the directory's [ as the start of the filter.

Write paths take no section: write("c.fits[compress]", x) (or a trailing (template.fits)) raises ValueError. Use write("c.fits", x, compress="RICE") for tile compression.


open_subset_reader()

Reusable reader for repeated cutout access on a single image HDU. Opens the file once; each call reads a region without re-opening.

torchfits.open_subset_reader(path, hdu=0, device="cpu")

hdu accepts an integer index or EXTNAME string (same as read_subset).

Returns: Context manager yielding a SubsetReader. Call reader(x1, y1, x2, y2) for each cutout.

with torchfits.open_subset_reader("mosaic.fits", hdu=0) as reader:
    stamp1 = reader(0, 0, 256, 256)
    stamp2 = reader(128, 128, 256, 256)

When to use

Many cutouts from the same large mosaic without reopening the file each time.


read_hdus()

Read multiple image HDUs from a single FITS file as a list of tensors.

torchfits.read_hdus(path, hdus, *, device="cpu", mmap=True, return_header=False)
Parameter Type Default Description
path str (required) FITS file path
hdus list[int or str] or tuple[int or str, ...] (required) Image HDU indices or EXTNAME strings
device str "cpu" Target device
mmap bool True Request the eligible memory-mapped input path
return_header bool False Also return a parallel list of Header objects

Returns: list[torch.Tensor], or (list[torch.Tensor], list[Header]) when return_header=True.

sci, wht, msk = torchfits.read_hdus("mef.fits", hdus=["SCI", "WHT", "MASK"])

Table Tensors (see Tables)

For reading and streaming catalog columns directly as PyTorch tensors:

Need API
Full / sliced column tensors table.read_torch
Streaming tensor chunks table.scan_torch
Arrow / Polars / where= table.read / table.read_polars
cols = torchfits.table.read_torch("catalog.fits", hdu=1, columns=["RA", "DEC"])
for chunk in torchfits.table.scan_torch("survey.fits", hdu=1, batch_size=100_000):
    print(chunk.keys())

read_batch()

Read the same HDU from multiple FITS files.

torchfits.read_batch(file_paths, hdu=0, device="cpu", *, strict=False)

Returns: list[torch.Tensor] — one tensor per input file (not a stacked batch), in input order. A path that cannot be read always raises, so the result list is never silently short. A missing file (FileNotFoundError) or HDU (HDUNotFoundError) keeps its type, with a note giving how many files read before it. With the default strict=False any other failure surfaces as a RuntimeError naming the path and how many files read before it; strict=True lets the original exception propagate unchanged.

tensors = torchfits.read_batch(["img1.fits", "img2.fits"], hdu=0)

read_batch_info()

Count how many of the given paths exist before a batch read. It does not open the files or compare shapes and dtypes.

torchfits.read_batch_info(file_paths)

Returns: dict with two integers: num_files (paths supplied) and existing_files (how many of them exist on disk).


open()

Multi-HDU context manager for low-level HDU/header access.

torchfits.open(path, mode="r")

Returns: HDUList context manager.

Header and HDU inspection (hdul[i].header, hdul.info(), TableHDURef metadata) uses a metadata-only reader that does not import PyTorch. Reading pixels or tensors (.data, .to_tensor()) opens the tensor-backed reader on first use.

with torchfits.open("mef.fits") as hdul:
    primary = hdul[0]  # TensorHDU
    sci = hdul["SCI"]  # TensorHDU by EXTNAME
    data = sci.data  # DataView (lazy)
    header = sci.header  # Header (dict-like)

Paths may include a CFITSIO image section (file.fits[10:100,20:200]); existence checks use the base path before [. Prefer hdu= / EXTNAME indexing over path HDU selectors (file.fits[1]) — those are not a certified torchfits open surface yet.

The open model is read-oriented

TensorHDU.data is a read-oriented DataView; HDUList has no flush() method or in-place write protocol. To create a modified file, construct the desired HDUs and call hdul.write(output_path, overwrite=...).

EXTNAME lookup returns first match

When indexing by EXTNAME (hdul["SCI"]), only the first HDU with that name is returned. The match follows read(path, hdu="SCI"): case-insensitive, surrounding blanks ignored, and an HDU without a matching EXTNAME matches on HDUNAME. "SCI" in hdul uses the same rule. FITS files with duplicate EXTNAMEs (e.g., multi-chip detectors with repeated SCI extensions) must use numeric indices for the second and subsequent occurrences.


read_header()

Read only the FITS header from an HDU.

torchfits.read_header(path, hdu=0)

Returns: Header (dict-like). Default hdu=0; pass hdu=None / "auto" to autodetect.

header = torchfits.read_header("image.fits", hdu=0)
print(header["EXPTIME"])  # e.g. 300.0

For bloated headers when you only need a few cards or the row count, prefer the skinny helpers below — they skip the full header dump.

read_nrows()

Table row count via CFITSIO fits_get_num_rows (no full header materialize).

torchfits.read_nrows(path, hdu=1)

Returns: int. Default hdu=1. Raises if the HDU is not a table.

n = torchfits.read_nrows("catalog.fits", hdu=1)

read_keys()

Selected header keywords via CFITSIO fits_read_keyword (no full header dump).

torchfits.read_keys(path, keys, hdu=0)

Returns: dict[str, Any]. Missing keys raise. Default hdu=0.

meta = torchfits.read_keys("image.fits", ["BITPIX", "NAXIS1", "NAXIS2"], hdu=0)

read_shape()

Image BITPIX + shape via CFITSIO image params (no full header).

torchfits.read_shape(path, hdu=0)

Returns: (bitpix, shape) with torch / row-major shape.

Skinny HDU type / count / EXTNAME

read_hdu_type() / read_num_hdus() / read_extname()

torchfits.read_hdu_type(path, hdu=0)  # "IMAGE" / "BINARY_TABLE" / ...
torchfits.read_num_hdus(path)
torchfits.read_extname(path, hdu=0)  # EXTNAME or None

Skinny colnames / table info

read_colnames() / read_table_info()

torchfits.read_colnames(path, hdu=1)
torchfits.read_table_info(path, hdu=1)  # {nrows, colnames, tforms}

One open each; no full header dump. Prefer these over read_header for counts, dims, and a handful of cards.

open_table_reader()

Open one table HDU once; call read_torch repeatedly without reopening.

torchfits.open_table_reader(path, hdu=1)

Returns: context manager yielding a handle with:

  • num_rows() → int
  • read_torch(columns=None, start_row=1, num_rows=-1, device="cpu") → dict[str, torch.Tensor]

No where= on the handle — use table.read_torch(..., where=...) for filtered reads (each call opens the file).

with torchfits.open_table_reader("catalog.fits", hdu=1) as reader:
    n = reader.num_rows()
    cols = reader.read_torch(columns=["RA", "DEC"])

Writes

write()

Write a tensor, numpy array, dict table, or HDUList to FITS.

torchfits.write(
    path,
    data,
    header=None,
    overwrite=False,
    compress=False,
    quantize=None,
    checksum=False,
)
Parameter Type Default Description
path str or PathLike (required) Output path
data Tensor, numpy array, dict, or HDUList (required) Data to write. Numpy arrays and tensors write an image HDU; dict writes a table.
header dict or Header or None None FITS header key-value pairs
overwrite bool False Overwrite an existing file. With False, a file that appears while writing is left untouched and FileExistsError is raised. A leading ! (CFITSIO clobber syntax) in path raises ValueError.
compress bool or str False True (= RICE_1), "gzip" (= GZIP_1), "rice", etc. Integer images are lossless with every algorithm. Float images are lossless only with GZIP_1; float GZIP_2, BZIP2_1, RICE_1 and HCOMPRESS_1 use CFITSIO default quantization (lossy, same as astropy/fitsio defaults). 64-bit integer images (int64, uint64) cannot be tile-compressed: compress= raises ValueError.
quantize None or str or dict None Opt-in robust BITPIX=16 pack for float images ("robust" or {"lo_q", "hi_q", "keep_zero"}). Default keeps native float.

uint16 images and PLIO_1

CFITSIO stores a uint16 image compressed with PLIO_1 as unsigned tile values without the BZERO = 32768 offset applied. torchfits, fitsio and other CFITSIO readers read it back exactly; astropy reads every value 32768 off. For files astropy users will open, compress uint16 images with RICE_1 or GZIP_1 instead:

import torch, torchfits

x = torch.tensor([[0, 1, 40000, 65535]], dtype=torch.uint16)
torchfits.write("u16.fits", x, compress="RICE_1", overwrite=True)
assert torch.equal(torchfits.read("u16.fits", hdu=1), x)

No tile compression for 64-bit integers

CFITSIO has no tile codec for BITPIX = 64. write(), write_tensor(), HDUList writes and insert_hdu()/replace_hdu() with compress= raise ValueError("tile compression does not support 64-bit integer images; ...") for an int64 or uint64 image, before anything is written. Write it uncompressed, or convert it first:

import torch, torchfits

x = torch.arange(16, dtype=torch.int64).reshape(4, 4)
torchfits.write("i32.fits", x.to(torch.int32), compress="GZIP_1", overwrite=True)

Skewed float → int16

Linear min→max packing onto int16 wastes codes on rare extremes. Prefer native float (quantize=None). When size forces int16, use quantize="robust" (default percentiles lo_q=0.1, hi_q=99.9 + clip) or a dict {"lo_q", "hi_q", "keep_zero"} — not global min/max. Same helper packs table columns via table.write(..., quantize=). See example_quantize_int16.py.

Errors from write()

Invalid arguments raise ValueError or TypeError unchanged (for example compress=3 raises TypeError). A failure inside the writer raises RuntimeError("Failed to write FITS file '<path>': ..."). Either way nothing is created or replaced at path.

The write() data-type convention

write() dispatches on the type of data: a dict (of tensors or arrays) always writes a table, a Tensor/numpy array always writes an image, and an HDUList writes its HDUs verbatim. A dict of 1-D equal-length columns and a 1-D image tensor are therefore easy to confuse — prefer write_tensor() for images and table.write() for tables when the payload shape is ambiguous.

Duplicate header keywords

The header= argument is a mapping, so it holds each keyword once. Repeatable keywords (HISTORY, COMMENT) can only be appended via the Header object API (add_history, add_comment) after opening the written file — passing {"HISTORY": ...} in header= sets a single card.

Header value types

A header value may be str, bool, int (any size: an integer beyond 64 bits is written as its exact literal), float, complex, or None (an undefined-value card) — every type read_header returns, so write(path, data, header=read_header(src)) round-trips. Anything else raises RuntimeError: Unsupported FITS header value type.

Header text must be printable ASCII

FITS restricts header values, comments and keywords to codes 32–126. Non-ASCII text is rejected with a ValueError naming the offending byte rather than silently stripped. The same applies to table column names and string column values. Reading stays forgiving — stray bytes already on disk are dropped so the file still opens.

Unsigned integers

uint16, uint32 and uint64 images and table columns are written with the FITS standard convention: signed storage plus BZERO/TZERO of 2**15, 2**31 or 2**63 (the uint64 offset is written as the exact integer card 9223372036854775808; files carrying the older float spelling still read as uint64). Reading such a file returns the unsigned dtype with exact values, the same as astropy and fitsio. int8 images and table columns use the signed-byte convention (unsigned byte storage plus BZERO/TZERO of -128) and read back as int8.

import torch, torchfits

x = torch.tensor([0, 2**63, 2**64 - 1], dtype=torch.uint64)
torchfits.write("u64.fits", x, overwrite=True)
assert torch.equal(torchfits.read("u64.fits"), x)

write_tensor()

Write a single PyTorch Tensor to a FITS image extension.

torchfits.write_tensor(
    path,
    tensor,
    header=None,
    overwrite=False,
    compress=False,
    quantize=None,
    checksum=False,
)
torchfits.write_tensor("out.fits", tensor, header={"OBJECT": "M31"}, overwrite=True)
torchfits.write_tensor("packed.fits", tensor, quantize="robust", overwrite=True)
torchfits.write_tensor("checked.fits", tensor, checksum=True, overwrite=True)

HDU Mutation

torchfits.insert_hdu(path, data, index=1, header=None, compress=False)
torchfits.replace_hdu(path, hdu, data, header=None, compress=False)
torchfits.delete_hdu(path, hdu, compress=False)

hdu is an index or an EXTNAME (matched case-insensitively, as in read()). Every other HDU keeps its index. An hdu (or insert index) that is not in the file raises HDUNotFoundError and leaves the file unchanged; insert_hdu accepts positions 0..len(file), the last one appending.

Only the inserted or replacing HDU is written, from the call's arguments. Every other HDU is copied byte for byte, so it keeps the structure it had before the edit: tile compression, storage BITPIX, BSCALE/BZERO/BLANK, scaled table columns with TSCAL/TZERO, and all other cards. compress= applies to the new HDU only. If the file carries CHECKSUM/DATASUM, the new HDU is stamped too. Bytes after the last HDU that do not start an extension (junk, zero or blank padding, which readers ignore) stay at the end of the file.

These are in-place edits: a file you cannot write (for example mode 0444) raises PermissionError and is left unchanged. write(path, ..., overwrite=True) instead replaces the whole file, which needs only a writable directory, and keeps the old file's mode.

# Add a RICE-compressed extension; HDUs 0 and 1 are left exactly as they were.
torchfits.insert_hdu(
    "mef.fits", image, index=2, header={"EXTNAME": "SCI2"}, compress="RICE"
)

HDU 0 changes structure only when the edit itself moves it:

  • insert_hdu(..., index=0) turns the old primary into an IMAGE extension (SIMPLE becomes XTENSION, PCOUNT/GCOUNT added, EXTEND dropped);
  • delete_hdu(path, 0) turns an IMAGE extension at index 1 into the primary (XTENSION becomes SIMPLE, EXTEND = T added, PCOUNT/GCOUNT dropped).

Data bytes are unchanged in both cases; only that HDU's checksum is restamped.

An edit that cannot land where asked raises ValueError and leaves the file unchanged:

  • a table at index 0 (insert_hdu(..., index=0), replace_hdu(path, 0, table), or delete_hdu(path, 0) when HDU 1 is a table) — a table cannot be the primary HDU;
  • compress= at index 0, or delete_hdu(path, 0) when HDU 1 is a tile-compressed image — a compressed image is a binary-table extension and cannot be the primary HDU;
  • insert_hdu(..., index=0) when the primary holds random groups;
  • delete_hdu(..., compress=...) — a delete writes no HDU, so there is nothing to compress.

Checksums

Stamp CFITSIO DATASUM/CHECKSUM while writing with checksum=True (applies to every HDU), or add them afterwards per HDU:

torchfits.write("img.fits", data, checksum=True)  # stamped at write time
torchfits.write_checksums(path, hdu=0)
torchfits.write_checksums(path, hdu="SCI")  # EXTNAME, case-insensitive as in read()
result = torchfits.verify_checksums(path, hdu=0)
# result: dict with "datastatus", "hdustatus", "ok", "present", and "status"
# ("ok" / "no_checksums" / "fail"). Missing stamps set present=False; ok
# stays True only because there was nothing to verify.

Cache Utilities

Layer Entry points Role
Disk downloads torchfits.cache.clear_cache(disk=True) / clear_all_caches() Everything under the configured cache root; in a TORCHFITS_REMOTE_CACHE directory set elsewhere, only the download files torchfits named there
Aggregate statistics torchfits.cache.get_cache_stats() In-process I/O cache counters
I/O metadata get_cache_performance(), clear_file_cache(...) In-process header / meta / data caches
Shared metadata (C++) clear_file_cache(..., cpp=True) Per-path metadata across private CFITSIO handles
torchfits.clear_file_cache(data=True, meta=True, hdu_types=True, stats=True, cpp=True)
torchfits.get_cache_performance()

torchfits.cache.get_cache_stats()
torchfits.cache.clear_cache()  # in-process only (default)
torchfits.clear_all_caches()  # in-process + configured disk cache

clear_cache() clears in-process data and metadata caches. Pass disk=True (or call root clear_all_caches()) to also remove downloaded files under the configured disk cache root (TORCHFITS_CACHE_DIR, the whole directory). A TORCHFITS_REMOTE_CACHE or TORCHFITS_SAMPLE_CACHE directory outside that root is never deleted: from the remote one only torchfits' own download files (<hash>.<ext> and their .partial / .lock siblings) are removed, and the sample one is left untouched, so pointing either at a shared directory such as $HOME cannot lose your files.

clear_file_cache keyword-only flags (all default True): data, meta, hdu_types, stats, cpp. Optional cpp_module= overrides the extension module used for the C++ clear. The legacy handles= flag was removed in 1.3.0: there is no shared CFITSIO handle pool to clear.

Advanced helpers on torchfits.io: cache_subsystem_policy(name) / clear_cache_subsystem(name) ("all" clears every subsystem).

Disk cache directories

Remote downloads and example samples use TORCHFITS_CACHE_DIR / TORCHFITS_REMOTE_CACHE / TORCHFITS_SAMPLE_CACHE (Environment variables). Those roots are separate from clear_file_cache / get_cache_performance. Remote-capable image, tensor, cube, and spectrum datasets accept cache_dir= to override the remote root; see Data module.